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The International Association for the Evaluation of Educational Achievement, 
known as IEA, is an independent, international consortium of national research 
institutions and governmental research agencies, with headquarters in Amsterdam. 
Its primary purpose is to conduct large-scale comparative studies of educational 
achievement with the aim of gaining more in-depth understanding of the effects of 
policies and practices within and across systems of education. 
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Preface 



The International Association for the Evaluation of Educational Achievement (IEA) is a 
non-profit, non-governmental international co-operative association of national research 
institutions and governmental research agencies from over 60 countries. Its secretariat is located 
in Amsterdam, the Netherlands, and it has a data processing and research center (DPC) in 
Hamburg, Germany. 

IEA conducts large-scale comparative studies of student achievement that have, as their 
particular focus, educational policies and practices in numerous countries around the world. 

The first IEA study was launched in 1958. Since then, almost 30 research studies, reporting on 
a wide range of topics and subject matters, have been conducted. Together, these studies have 
contributed to a deeper understanding of educational phenomena within individual countries 
and within a broad international context. 

The International Civic and Citizenship Study (ICCS) is the third IEA project investigating the role 
of schooling in preparing young people for their roles as citizens in society. The first study in 
this area — the Study of Civic Education — began in 1971. This project, which introduced civics 
and civic education into the IEA research agenda, challenged some of the conceptual and 
methodological constraints associated with international comparisons in this area of education. 

The second Civic Education Study (CIVED), undertaken in 1999, focused on investigating school 
experiences in the context of changes that occurred in the “real world” of the political and 
social life of nations in the early 1990s. These changes included, on the one hand, the collapse 
of communism in Central-Eastern Europe and the appearance of new democracies and, on 
the other hand, a lessening of interest and participation in public life by citizens of the older 
democracies. 

Almost 10 years on, global change has again prompted a new survey of civic and citizenship 
education. Amongst other considerations, the growing impact of the processes of globalization, 
external threats to civic societies and their freedoms, and the limited interest and involvement 
of young generations in public and political life have stimulated renewed reflection on the 
meanings of citizenship and the roles of and approaches to civic and citizenship education. 

The IEA General Assembly agreed that it was timely to address these emerging concerns and 
initiated ICCS in order to collect empirical evidence that individual countries and international 
organizations can use to improve policy and practice in this area. 

The aim of ICCS is to report on student achievement on a test of conceptual knowledge and 
understandings in civic and citizenship education. It also intends to collect and analyze data 
about student dispositions and attitudes relating to civic and citizenship education. Because 
ICCS builds on CIVED (1999), it will give those education systems that participated in the 
earlier study an opportunity to generate indicators (trend data) of national across-time progress 
in student achievement. However, ICCS also offers opportunity to pursue new targets in this 
field of education. A major innovation is that ICCS will attempt, through the introduction of 
regional modules, to address specific regional issues in civic and citizenship education. Three 
regional modules (Asian, European, and Latin American) will provide data that will complement 
the information collected relative to the core part of the study. 

International studies of the scale of ICCS rely on the contributions and collaboration of many 
individuals and input from many countries. This publication contains the ICCS assessment 
framework, which provides the blueprint for the assessment of the outcomes of civic and 
citizenship education. The publication has been a collaborative process involving numerous 
individuals and groups, including the project advisory committee (PAC), national research 
coordinators (NRCs) from over 30 countries, and other experts. Input from these groups 
and individuals was sought in the early stages of this study, and strategies were developed to 
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encourage ICCS participating countries to contribute to the development of the assessment 
framework, the design of the study, and the study’s instruments. 

Projects of this magnitude require a significant financial commitment from IEA and its partners. 
Critical core funding for ICCS has been provided by the following: the European Commission 
Directorate-General for Education and Culture in the form of a grant to the European countries 
participating in the project; the Inter-American Development Bank through SREDECC (the 
Regional System for the Evaluation and Development of Citizenship Competencies), which 
supported a number of Latin American countries; and the ICCS participating countries. 

The success of this project depends on sound management practices. Three partner institutions, 
in co-operation with the IEA Secretariat, the IEA DPC, and the NRCs, are responsible for the 
study’s organization. They are the Australian Council for Educational Research (ACER), as lead 
institution, the National Foundation for Educational Research (NFER) in the United Kingdom, and 
the Laboratorio diPedagogia sperimentale (LPS) at the Roma Tre University in Italy. 

I express my sincere thanks to the study leaders: John Ainley, Julian Fraillon, and Wolfram 
Schulz from ACER; David Kerr from NFER; and Bruno Losito from LPS. I also thank the 
researchers from the consortium institutions: Suzanne Mellor and Naoko Tabata from ACER; 
Gabriella Agrusti from LPS; and Joana Lopes from NFER. 

My special thanks go to the members of the PAC for their thoughtful reviews of earlier versions 
of the assessment framework. I particularly thank Judith Torney-Purta (University of Maryland), 
previously the leader and chair of the steering committee for CIVED. The IEA publications and 
editorial committee (PEC) suggested improvements to earlier versions of the framework, and 
Paula Wagemaker edited the document. 

Finally, I wish to express my appreciation for the contribution made by the NRCs. They are the 
core of each IEA study because they assure its relevance to the research, policy, and educational 
practice relevant to the area of investigation. 



Hans Wagemaker 
EXECUTIVE DIRECTOR IEA 
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Purpose of the study 

The purpose of the International Civic and Citizenship Education Study (ICCS) is to investigate, in 
a range of countries, the ways in which young people are prepared and consequently ready 
and able to undertake their roles as citizens. In pursuit of this purpose, the study reports on 
student achievement in a test of conceptual understandings and competencies in civics and 
citizenship. It also collects and analyzes, as additional outcome variables, data about student 
activities, dispositions, and attitudes related to civic and citizenship education. The collection 
of contextual data will help explain variation in the outcome variables. The proposal builds on 
the previous IEA studies of civic education, acknowledges the need for a new study, and is a 
response to the challenge of educating young people in changed contexts of democracy and 
civic participation. 

Background to the study 

Previous IEA studies 

The International Association for the Evaluation of Educational Achievement (IEA) has already 
carried out two international surveys of student characteristics in the domain of civic education. 
The first such study (Torney, Oppenheim, & Farnen, 1975) was conducted as part of the so- 
called Six Subject Study, with data collected in 1971 (for a summary, see Walker, 1976). The 
second study, the IEA Civic Education Study (CIVED), was carried out in 1999. It was designed 
to strengthen the empirical foundations of civic education by providing up-to-date information 
about the civic knowledge, attitudes, and actions of 14-year-olds. 

CIVED had a twin focus on school-based learning and on opportunities for civic participation 
outside the school. CIVED concentrated on three civic-related domains: democracy/ citizenship; 
national identity/international relations; and social cohesion /diversity. It was argued that the 
research outcomes from a study focusing on these domains would be particularly useful to 
policymakers involved in designing or redesigning curricula and preparing teachers. 

CIVED was successful in meeting its aims and objectives. Phase 1 produced a detailed series of 
national case studies from the 24 participating countries (Torney-Purta, Schwille, & Amadeo, 
1999). Phase 2 produced two data-rich international reports, the first on the results from the 
mandatory standard population of 14-year-olds in 28 participating countries (Torney-Purta, 
Lehmann, Oswald, & Schulz, 2001), and the second from the 16 countries that surveyed an 
older, optional population of 16- to 18-year-olds (Amadeo, Torney-Purta, Lehmann, Husfeldt, 

& Nikolova, 2002). CIVED’s findings have had a considerable influence on policy and 
educational practice in civic and citizenship education across the world, in both participating 
and non-participating countries, and have also influenced further research in this area (Kerr, 
Ireland, Lopes, & Craig, with Cleaver, 2004; Mellor 8c Prior, 2004; Menezes, Ferreira, Carneiro, 
& Cruz, 2004). 

Recent changes in the world context of civics and citizenship 

In the 10 years since CIVED investigated civic education, the conditions relevant to civics and 
citizenship (especially governance and among-nation relations) have undergone considerable 
change across the globe. The world context and thus the context in relation to both citizenship 
and policy and practice in civic and citizenship education have changed. 
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The catalyst of major political change that swept across the world in the late 1980s and 1990s, 
and which triggered CIVED, has developed further: 

• The last decade has seen a significant change in external threats to civil societies. Terrorist 
attacks and the ensuing debates about the response civil societies should take have resulted 
in greater importance being attached to civic and citizenship education. This shift has been 
accompanied by a growing recognition of the role that civic and citizenship education can 
play in response to these changes (see Ben Porath, 2006; IDEA, 2006). 

• Observation in many developing countries, and particularly those in the Latin American 
region, indicates that people are giving greater value to democracy as a system of 
government than in the past, but that increasing social and economic inequalities 

are simultaneously threatening the continuation of democratic government. Studies 
have highlighted the consensus in many developing countries of the importance of 
strengthening and promoting citizenship education among both young people and adults 
in order to make democracy more sustainable (see Cox, Jaramillo, & Reimers, 2005; 
Reimers, 2007; USAID, 2002). 

• There has been a notable increase in the importance of non-governmental groups from 
civil society serving as alternative vehicles through which active citizenship can unite 
citizens with common purpose. New forms of social movement participation are due to a 
variety of different purposes, ranging from religious motivation to issues such as protection 
of human rights or protection of the environment (see Scheufele, Nisbet, & Brossard, 

2002; Wade, 2007). 

• The trend toward more profound cultural changes due to the modernization and 
globalization of societies, more universal access to new media such as the internet, 
increasing consumer consumption, and transformation of societal structures (individualism) 
has continued over the last decade (see Branson, 1999; Rahn, 2004). 

Consequently, interest in civic and citizenship education remains, or has become, high in many 
countries, especially in those with democratic forms of government, but also in those where the 
establishment of democracy has yet to be achieved. 

Linked to these changes is a change in views with regard to the appropriateness of traditional 
views of citizenship. This development has led to a revisiting of concepts and practices 
associated with the four dimensions of citizenship: rights and responsibilities, access, belonging, 
and other identities (see Banks, 2004; Kymlicka, 2001; Macedo, 2000). Current debates 
include discussions about concepts of national identity, how national identity can be identified, 
and what might be done to confirm national identity. 

Low participation in governance and social dialectic by voters, or populations, is of particular 
relevance in many countries, and concerns are growing about the lack of interest and 
involvement of young people and young adults in public and political life (Curtice & Seyd, 
2003; Putnam, 2000). However, while young people may reject political practices, the same 
does not necessarily occur with their endorsement of political values like solidarity, equity, and 
tolerance. There is also some evidence that young people are becoming increasingly involved in 
alternative forms of participation (Torney-Purta et al., 2001). 
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Developments in the research of civic and citizenship education 

Because of these developments, experts and educational policymakers in many countries are 
reflecting on the meaning of citizenship, resulting in an increased focus on civic and citizenship 
education in schools. There has been an increase in the number of countries and regions 
interested and involved in progressing civic and citizenship education. Such countries have 
become more proactive, and research in recent years has provided insights into: 

• The gap between the intended and the implemented curriculum (Birzea et al., 2004; 
Eurydice, 2005); 

• The conceptualization of citizenship in schools with respect to curriculum, school culture, 
and the wider community (Kerr et al., 2004); 

• Approaches to civic and citizenship education, with an emphasis on active and experiential 
teaching and learning (Potter, 2002); and 

• Those factors that support effective citizenship education (Craig, Kerr, Wade, 8c Taylor, 
2005). 

This growing policy and evidence base has helped facilitate increased collaboration and sharing 
of expertise within and across countries and regions. Generally, the environment for civic and 
citizenship education has changed considerably since the late 1980s with respect to the scale 
and complexity of the challenges facing democracy and citizenship. There is a need to update 
the empirical evidence of this new environment in civic and citizenship education. 

Research questions, design, and instrumentation 

The key research questions for the study concern student achievement, dispositions to engage, 
and attitudes related to civic and citizenship education. The variables necessary to analyze these 
research questions can be located in the contextual framework. 

RQJ What variations exist between countries, and within countries, in student achievement in conceptual 
understandings and competencies in civics and citizenship? 

This research question concerns the distribution of outcome variables across 
participating countries (at the country level) and within these countries. Analysis to 
address this research question focuses on the distribution of student achievement based 
on test data and involves single- and multi-level perspectives (through the use of, for 
example, models of variance decomposition). 

RQJ What changes in civic knowledge and engagement have occurred since the last international 
assessment in 1999? 

This research question is mainly concerned with analyzing trends from CIVED to ICCS 
and will be limited to data from countries participating in both assessments. Analysis 
focuses at the level of participating countries on changes in overall civic knowledge and 
indicators of civic engagement and attitudes. Country-level factors (recent curriculum 
changes, reforms) can be used when interpreting possible changes across time. 

RQJ What is the extent of interest and disposition to engage in public and political life among adolescents 
and which factors within or across countries are related to it? 

This research question addresses the issue of engagement versus apathy, with indicators 
of civic participation compared within and across countries and related to explanatory 
variables at various levels. Both characteristics and process-related variables at the levels 
of school/ classroom and home environment are used to explain variation in outcome 
variables. 



ICCS ASSESSMENT FRAMEWORK 



9 



RQjt What are adolescents’ perceptions of the impact of recent threats to civil society and of responses to 
these threats on the future development of that society? 

Analysis is based on student comprehensions of the relationship between securing 
societies and safeguarding civil liberties, and on student attitudes toward citizenship 
rights. Factors located at the country level (recent developments, liberal traditions) might 
be of particular importance for the analysis. 

RQJ> What aspects of schools and education systems are related to achievement in and attitudes to civics 
and citizenship, including: 

(a) general approach to civic and citizenship education, curriculum, and/ or program content 
structure and delivery? 

The analysis requires additional data to be collected at the national level on curriculum 
and programs as well as from reports from the school and teacher questionnaires. 

Both background variables and factors related to the process of civic and citizenship 
learning at the country level and the school/ classroom level are therefore of particular 
importance for the analysis. 

(b) teaching practices, such as those that encourage higher order thinking and analysis in relation to 
civics and citizenship? 

Analysis will be based on data about student perceptions of and teacher reports on 
instructional practices and will involve variables related to the learning process collected 
from schools, teachers, and individual students. 

(c) aspects of school organization, including opportunities to contribute to conflict resolution, 
participate in governance processes, and be involved in decisionmaking? 

The analysis requires data on student perceptions of school governance and reports 
from school principals or teachers. It involves variables that relate to the opportunities 
students have to participate within school and that are collected from students, teachers, 
and schools. 

RQJ> What aspects of student personal and social background, such as gender, socioeconomic background, 
and language background, are related to student achievement in and attitudes toward civic and 
citizenship education? 

The analysis will rely on student background variables, collected through the student 
questionnaire and related to the individual students and the home environment. These 
factors will be used to explain variation in outcome variables assessed in the test and the 
student perceptions questionnaire. 

The research questions played an important role in shaping the design of the study and its 
instrumentation and in guiding the development of the assessment framework. 

The student population surveyed includes students in Grade 8 (on average including students 
who are approximately 14 years of age), provided that the average age of students in Grade 8 is 
13.5 years or above. Where the average age of students in Grade 8 is under 13.5 years, Grade 9 
is defined as the target population. 1 In each sampled school, intact classrooms are selected, and 
all students in a class are assessed for the ICCS survey. 

The population for the ICCS teacher survey is defined as all teachers teaching regular 
school subjects to the students in the target grade at each sampled school. It includes only 
those teachers teaching Grade 8 during the testing period and employed at school from 
the beginning of the school year. Fifteen teachers are randomly selected from each school 
participating in the ICCS survey. 

1 An option to include Grade 9 as an additional year level is offered to countries that participated in CIVED using their 
Grade 9 population and who are assessing Grade 8 in ICCS. 
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An important feature of ICCS is the establishment of regional modules. Regional modules 
compromise groups of countries from the same geographic region that together administer 
additional instruments to assess region-specific aspects of civic and citizenship education. Three 
regional modules have been implemented as part of ICCS for participating countries in the 
regions of Europe, Latin America, and Asia respectively. 

The following instruments are administered as part of the ICCS survey: 

• An international cognitive student test consisting of items measuring students’ civic knowledge 
and ability to analyze and reason. 

• A student questionnaire consisting of items measuring student background variables and 
student perceptions and behaviors. 

• A regional student instrument , administered after the international student assessment and 
possibly consisting of region-specific cognitive and questionnaire-type items. This 
instrument will only be used in countries participating in a regional module. 

• A teacher questionnaire, administered to selected teachers teaching any subject in the target 
grade. It gathers information about teacher background variables and teachers’ perceptions of 
factors related to the context of civic and citizenship education in their respective schools. 

• A school questionnaire, administered to school principals of selected schools to capture 
school characteristics and school-level variables related to civic and citizenship 
participation. 

• An online national contexts survey, completed by national center experts with regard to the 
structure of the education systems, the status of civic and citizenship education in the 
national curriculum, and recent developments. The data obtained from this survey assist 
with the interpretation of the results from the student, school, and teacher questionnaires, 
and provide a description of the context for civic and citizenship education in each 
country. 

The assessment framework provides a conceptual underpinning for the international 
instrumentation for ICCS and has also been a point of reference for the development of 
regional instruments. 

Characteristics and structure of the ICCS assessment framework 

Given the central role of the assessment framework in the process of instrument development, it 
has been important to: 

• Maintain a strong connection to the constructs used in the IEA CIVED survey of 1999; 

• Reflect contemporary research understandings of manifestations of civic and citizenship 
education in school students; 

• Meet the needs of participating countries; 

• Address the research questions outlined in the ICCS proposal for the IEA General 
Assembly of 2005; 

• Include only content that can be measured; 

• Compile content descriptors that are agreed to be significant and discrete and that describe 
the breadth of civic and citizenship education for school students; and 

• Address the contexts within which civic and citizenship education takes place. 

The assessment framework consists of two parts: 

• The civic and citizenship framework outlines the aspects to be addressed when collecting the 
outcome measures through the cognitive test and the student perceptions questionnaire. 

• The contextual framework provides a mapping of context factors that might influence 
outcome variables and explain their variation. 
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Civic and Citizenship Framework 



Overview 

Continuities between CIVED and ICCS 

Key conceptual continuities 

The overall model for CIVED is represented as an octagon that gives detail to the “nested 
context for young people’s thinking and action in the social environment” (Torney-Purta et al., 
2001, p. 21). The detailed CIVED model is reproduced as Figure 1. 

The CIVED theoretical model places the individual student at its center, with the student 
influenced by “agents” of socialization. The model is based on the assumption that students’ 
“learning about citizenship” is not limited to teachers explicitly instructing young people about 
their rights and duties (Torney-Purta et al., 2001). 

Figure 1: Model of the IEA Civic Education Study (CIVED) 




The ICCS assessment framework thus reflects the pivotal assertion of the CIVED model that 
the individual student exists as the central agent in their civic world, with both an influence on 
and being influenced by their multiple connections with their civic communities. Consequent to 
this is the assertion posited in CIVED, and further reflected by the ICCS assessment framework, 
that young people learn about civics and citizenship through their interactions with their 
multiple civic communities and not just through formal classroom instruction (Torney-Purta et 
al., 2001). 

Construct operationalization continuities 

The CIVED conceptual model was operationalized to collect student outcomes data, using a 
cognitive test and questionnaire instruments with a range of question types relating to civic and 
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citizenship content in three domains. The three domains of the CIVED conceptual model of 
civics and citizenship are: 

• Domain I: Democracy/ Citizenship 

• Domain II: National Identity/International Relations 

• Domain III: Social Cohesion/Diversity. 

A set of the secure CIVED trend cognitive items and items from some of the attitude and 
concept CIVED scales are included in the ICCS assessment instruments. 

Representing civics and citizenship in the ICCS assessment framework 

The ICCS assessment framework is organized around three dimensions: a content dimension 
specifying the subject matter to be assessed within civics and citizenship; an affective- 
behavioral dimension that describes the types of student perceptions and activities that are 
measured; and a cognitive dimension that describes the thinking processes to be assessed. 

The four content domains in the ICCS assessment framework are: 

• Content Domain 1 : Civic society and systems 

• Content Domain 2: Civic principles 

• Content Domain 3: Civic participation 

• Content Domain 4: Civic identities. 

It is important to distinguish the different types of student perceptions and behaviors relevant 
within the context of civics and citizenship. For this purpose, four affective-behavioral domains 
are identified in the ICCS assessment framework: 

• Affective-behavioral Domain 1: Value beliefs 

• Affective-behavioral Domain 2: Attitudes 

• Affective-behavioral Domain 3: Behavioral intentions 

• Affective-behavioral Domain 4: Behaviors. 

Similar to the domains within the assessment framework for TIMSS (Mullis, Martin, Ruddock, 
O’Sullivan, Aroa, & Erberber, 2005), the cognitive domains in the ICCS assessment framework 
define the cognitive processes assessed with test items. The two cognitive domains in the ICCS 
framework are: 

• Cognitive Domain 1 : Knowing 

• Cognitive Domain 2: Reasoning and analyzing. 

Mapping assessment domains to the assessment instruments 

The ICCS assessment of the outcomes of civic and citizenship education comprises two 
instruments: 

• A student test measuring cognitive processes 

• A student questionnaire measuring perceptions and behaviors. 

The data from the cognitive test will be used to construct a scale of civic and citizenship 
knowledge and understandings as described by the two cognitive domains, and representing 
the substance of the four content domains. 

The data from the student questionnaire will be used to articulate perceptions constructs 
pertaining to the four affective-behavioral domains and relating to the substance of the four 
content domains. The amount and type of assessment information accessed by each instrument 
will vary across the four content domains. 
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Data dealing with active citizenship behaviors will be used as indicators of active citizenship 
and as possible explanatory variables of civic and citizenship achievement. Indicators of student 
activities are also important context variables and are therefore also included in the contextual 
framework. 

Structures and key terms in the ICCS assessment framework 

Structure of the content domains 

The four content domains of the ICCS assessment framework share the following structures: 
Sub-domain This refers to a substantive or contextual component of a content domain. 



Key concept This refers to concepts and processes common to sub-domains within a 
given content domain. 

In short, each content domain is divided into sub-domains, and each sub-domain consists of 
one or more aspects. The key concepts can be expressed within the contexts of any of the 
sub-domains. Figure 2 illustrates the structure of the content domains in the ICCS assessment 
framework. 

Figure 2: The structure of the ICCS assessment framework content domains 



The sub-domains are described if they include sufficient discrete content to 
warrant individual definition and articulation. This model anticipates some 
overlap between the sub-domains within each domain. 



Aspect This refers to specific content regarded as largely situated within a given 

sub-domain. 



Domains (1-4) 



are 

divided 

into 




Sub-domains 





Key Concepts 



Aspects 

• ASdla 

• ASdlb 



• ... 

• ... 



• KC1 



Sdl 




• KCn 



• ... 




• ASdna 

• ASdnb 



Note: KC = Key concept; Sd = Sub-domain; ASd = Aspect. 
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Structure of the affective-behavioral domains 

The four affective-behavioral domains are described in terms of the types of student perceptions 
and behaviors relevant with respect to the civic and citizenship content domains. 

Structure of the cognitive domains 

The two cognitive domains are each defined in terms of the cognitive processes that comprise 
them. 

Key terms 

The ICCS assessment framework uses a set of key terms. Definitions of these follow. Note that 
the exact definitions of many of the terms used in the framework are the subject of ongoing 
and vigorous academic dialogue. The definitions of the key and domain-specific terms in this 
framework have been constructed to support consistent understandings of the framework’s 
contents across the broad range of countries participating and interested in ICCS. 



Community 


A group of people who share something in common (for example, 
history, values, loyalties, a common goal). In this framework, community 
membership includes membership based on externally defined criteria 
relating to the function of the community (such as attending a school as 
a student) and membership defined by individuals’ own belief of their 
membership (such as through identification with “like-minded” people 
regarding a political or social issue). 2 


Society 


A community defined by its geographical territory and within which the 
population shares a common culture (which may comprise and celebrate 
multiple and diverse ethnic or other communities) and way of life under 
conditions of relative autonomy, independence, and self-sufficiency. 


Citizenship 


1. The legal status of being a citizen. 




2. The fact of individuals’ participation, or lack of participation, in their 
communities. The term “citizenship,” unlike the term “active citizenship,” 
does not assume certain levels of participation. 


Civil 


Refers to the sphere of society in which the shared connections between 
people are at a level larger than that of the extended family but do not 
include connections to the state. 


Civic 


Refers to any community in which the shared connections between 
people are at a level larger than that of the extended family (including 
the state). Civic also refers to the principles, mechanisms, and processes of 
decisionmaking, participation, governance, and legislative control that exist 
in these communities. 



2 Note that a community may still contain any level of diversity. 
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Civic and citizenship content domains 

The first content domain, civic society and systems, comprises the mechanisms, systems, and 
organizations that underpin societies. The second domain, civic principles, refers to the shared 
ethical foundations of civic societies. Civic participation deals with the nature of the processes 
and practices that define and mediate the participation of citizens in their civic communities 
(often referred to as active citizenship). The ICCS assessment framework recognizes the 
centrality of the individual citizen through the civic identities domain. This domain refers to the 
personal sense an individual has of being an agent of civic action with connections to multiple 
communities. Together, these four domains describe the civic and citizenship content to be 
assessed in ICCS. 

Content Domain 1: Civic society and systems 

Civic society and systems focuses on the formal and informal mechanisms and organizations that 
underpin both the civic contracts that citizens have with their societies and the functioning of 
the societies themselves. The three sub-domains of civic society and systems are: 

• Citizens 

• State institutions 

• Civil institutions. 

Citizens 

Citizens focuses on the civic relationships between individuals and groups of citizens and their 
societies. The aspects of this sub-domain relate to knowledge and understanding of: 

• Citizens’ and groups’ assigned and desired roles within their civic society 

• Citizens’ and groups’ assigned and desired rights within their civic society 

• Citizens’ and groups’ assigned and desired responsibilities within their civic society 

• Citizens’ and groups’ opportunities and abilities to support the ongoing development of their civic 
society. 

State institutions 

State institutions focuses on those institutions central to the processes and enacting of civic 
governance and legislation in the common interest of the people they represent and serve. 

The aspects of this sub-domain are: 

• Legislatures / parliaments 

• Governments 

• Supranational/ intergovernmental governance bodies 

• Judiciaries 

• Law enforcement bodies 

• National defense forces 

• Bureaucracies (civil or public services) 

• Electoral commissions. 
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Civil institutions 

Civil institutions focuses on those institutions that can mediate citizens’ contact with their state 
institutions and allow citizens to actively pursue many of their roles in their societies. 

The aspects of this sub-domain are: 

• Religious institutions 

• Companies/ corporations 

• Trade unions 

• Political parties 

• Non-governmental organizations (NGOs) 

• Pressure groups 

• The media 

• Schools 

• Cultural / special-interest organizations. 



Key concepts 



Power/ authority 


Listed together as concepts dealing with the nature and consequences 
of the right or capacity of bodies or individuals to make binding 
decisions on behalf of others that these others are then required to 
accept and adhere to. 


Rules/ law 


Listed together as the explicit and implicit prescriptions for behavior. 
Rules are those prescriptions that are not required to be, and are 
therefore not, enforced by a sovereign body. Laws are considered to 
be those prescriptions enforced by a sovereign body. 


Constitution 


The fundamental rules or laws of principle governing the politics of a 
nation or sub-national body. 


Governance 


The act and the processes of administering public policy and affairs. 


Decisionmaking 


The formal and informal processes by which decisions are made 
within and among civil and state institutions. 


Negotiation 


The processes that underpin and are evident in negotiation, and the 
use and necessity of negotiation as a means of decisionmaking. 


Accountability 


The requirement for representatives to answer to those they represent 
about the representatives’ conduct of their duties and use of their 
powers. Accountability includes the assumption that representatives 
are able to accept responsibility for their failures and to take action to 
rectify them. 


Democracy 


The ICCS assessment framework accepts the broadest definition 
of democracy “as rule by the people.” This definition refers both 
to democracy as a system of governance and to the principles of 
freedom, equity, and social cohesion 3 that underpin democratic 
systems and guarantee respect for and promotion of human rights. 
Both representative democratic systems (such as national parliaments) 
and direct democratic systems (such as those in some local-community 
or school organizations) can be examined as democratic systems under 
the definition of democracy used in this framework. 



3 See Civic and Citizenship Content Domain 2, page 19. 
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Sovereignty 



Nation-building 



Statelessness 



Franchise/ voting 



The economy 



The welfare state 



Treaties 

Sustainable 

development 



Globalization 



The claim of each individual state/ nation to have the ultimate power 
in making political decisions relevant to that state/ nation and the 
recognition that this power underpins the operation and viability of 
international organizations, agreements, and treaties. 

The process of developing among the people of a nation some form 
of a unified sense of national identity, with the aim of fostering 
long-term harmony and stability. Within the parameters of the ICCS 
assessment framework, nation-building is assumed to be a dynamic 
ongoing process in all nations rather than a process associated only 
with newly independent nations. 

The circumstances of people who do not have any legal bond of 
nationality or citizenship with any state. Included in this concept are 
the causes and consequences of statelessness. 

Listed together, these concepts refer to the rights, responsibilities, and 
expectations of people to vote in formal and informal settings. These 
concepts also refer, more broadly, to issues associated with voting and 
voting processes, such as compulsory and voluntary voting and secret 
ballots. 

Systems governing the production, distribution, and consumption of 
goods and services within states, including industrial regulation, trade, 
taxation, and social welfare. 

The role of a government in providing for the social and economic 
security of its people through support such as health care, pensions, 
and social welfare payments and benefits. 

Binding agreements under international law entered into by eligible 
bodies such as states and international organizations. 

Development that meets the needs of the present without 
compromising the ability of future generations to meet their own 
needs. Factors that can be considered in terms of sustainable 
development include environmental protection, economic 
development, social equality, and social justice. 

The increasing international movement of commodities, money, 
information, and people; and the development of technology, 
organizations, legal systems, and infrastructures to allow this 
movement. The ICCS assessment framework acknowledges that 
a high level of international debate surrounds the definition, 
perceptions, and even the existence of globalization. Globalization has 
been included in the framework as a key concept for consideration 
by students. The definition is not a statement of belief about the 
existence or merits of globalization. 
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Content Domain 2: Civic principles 

Civic principles focuses on the shared ethical foundations of civic societies. The framework 
regards support, protection, and promotion of these principles as civic responsibilities and as 
frequently occurring motivations for civic participation by individuals and groups. The three 
sub-domains of civic principles are: 

• Equity 

• Freedom 

• Social cohesion. 



Equity 

Equity focuses on the principle that all people have the right to fair and just treatment and that 
protecting and promoting this equity is essential to achieving peace, harmony, and productivity 
within and among communities. The principle of equity is derived from the notion of 
equality — that all people are born equal in terms of dignity and rights. 

Freedom 

Freedom focuses on the concept that all people should have freedom of belief, freedom of 
speech, freedom from fear, and freedom from want as articulated in the United Nations 
Universal Declaration of Human Rights (United Nations, 1948). Societies have a responsibility 
to actively protect the freedom of their members and to support the protection of freedom in all 
communities, including those that are not their own. 

Social cohesion 

Social cohesion focuses on the sense of belonging, connectedness, and common vision that exists 
amongst the individuals and communities within a society. When social cohesion is strong, 
there is active appreciation and celebration of the diversity of individuals and communities 
that comprise a society. It is acknowledged (in regard to this sub-domain) that manifestations 
of social cohesion vary between societies, that there may be tensions within societies between 
social cohesion and diversity of views and actions, and that the resolution of these tensions is 
an ongoing area of debate within many societies. 



Key concepts 



Concern for the The concept that the ultimate goal of civic and community action is to 
common good promote conditions that advantage all members of the community. 

Human rights A form of inalienable moral entitlement that, for the purpose of the ICCS 
assessment framework, is framed by the contents of the United Nations 
Universal Declaration of Human Rights (United Nations, 1948). 



Empathy Intellectually or emotionally taking the role or perspective of others. 



Respect 



The concept that all people are to be valued because they are human. 



Social justice The distribution of advantage and disadvantage within communities. 



Inclusiveness The concept that communities have a responsibility to act in ways that 

support all their members to feel valued as members of those communities. 



Equality 



The notion that all people are born equal in terms of dignity and rights 
regardless of their personal characteristics (such as gender, race, religion). 
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Content Domain 3: Civic participation 

Civic participation refers to the manifestations of individuals’ actions in their communities. Civic 
participation can operate at any level of community and in any community context. The level 
of participation can range from awareness through engagement to influence. The three sub- 
domains of civic participation are: 

• Decisionmaking 

• Influencing 

• Community participation. 

Decisionmaking 

Decisionmaking focuses on active participation that directly results in the implementation of 
policy or practice regarding the individual’s community or a group within that community. The 
aspects of this sub-domain are: 

• Engaging in organizational governance 

• Voting. 

Influencing 

Influencing focuses on actions aimed at informing and affecting any or all of the policies, 
practices, and attitudes of others or groups of others in the individual’s community. The aspects 
of this sub-domain are: 

• Engaging in public debate 

• Engaging in demonstrations of public support or protest 

• Engaging in policy development 

• Developing proposals for action or advocacy 

• Selective purchasing of products according to ethical beliefs about the way they were produced (ethical 
consumption / ethical consumerism) 

• Corruption. 

Community participation 

Community participation focuses on participation, with a primary focus on enhancing one’s 
connections with a community, for the ultimate benefit of that community. The aspects of this 
sub-domain are: 

• Volunteering 

• Participating in religious, cultural, and sporting organizations 

• Keeping oneself informed. 

Key concepts 



Civic involvement The concept that civic communities benefit from the active involvement 
of their citizens and that therefore there is an onus on civic communities 
to facilitate the right of active citizenship and an onus on citizens to 
participate actively in their civic communities. 

Civic self-efficacy Individuals’ own judgments of their capacity to complete courses of 
action that will influence their civic communities. 
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Co-operation/ 

collaboration 



Negotiation/ 

resolution 



The concept that communities benefit most when their members act 
together in pursuing the common goals of the community. (This 
definition allows for disagreement within communities about the best 
way to achieve their goals.) 

The concept that peaceful resolution of differences is essential to 
community wellbeing and that negotiation is the best way to attempt to 
reach resolutions. 



Engagement The concept that citizens need to concern themselves with issues and 
information in their communities in order to participate effectively. 



Content Domain 4: Civic identities 

Civic identities includes the individual’s civic roles and perceptions of these roles. As was the 
case with the CIVED model, ICCS assumes that individuals both influence and are influenced 
by the relationships they have with family, peers, and civic communities. Thus, an individual’s 
civic identity explicitly links to a range of personal and civic interrelationships. This framework 
asserts and assumes that individuals have multiple articulated identities rather than a single- 
faceted civic identity. 

Civic identities comprises two sub-domains: 

• Civic self-image 

• Civic connectedness. 

Civic self-image 

Civic self-image refers to individuals’ experiences of their place in each of their civic 
communities. Civic self-image focuses on individuals’ civic and citizenship values and roles, 
individuals’ understanding of and attitudes toward these values and roles, and individuals’ 
management of these values and roles whether they are in harmony or in conflict within each 
of these people. 

Civic connectedness 

Civic connectedness refers to the individual’s sense of connection to their different civic 
communities and to the different civic roles the individual plays within each community. 

Civic connectedness also includes the individual’s beliefs about and tolerance of the levels of 
diversity (of civic ideas and actions) within and across their communities, and their recognition 
and understanding of the effects of the range of civic and citizenship values and belief systems 
of their different communities on the members of those communities. 

Key concepts 

Civic self-concept Individuals’ views of themselves as citizens in their civic 
communities. This view includes individuals’ sense of the 
communities to which they belong and their capacity to identify the 
nature and parameters of their civic roles in their communities. 

Multiplicity Individuals’ sense of the range of different roles and potentials they 

have within and across their different communities. Included in this 
concept is the understanding that an individual’s roles and potentials 
are ever changing and that these depend on the context of each 
separate community connection. 
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Diversity 


Individuals’ sense and level of acceptance of the range of people and 
viewpoints that exist within and across their communities. 


Cultures/ location 


Individuals’ sense of the value and place of the cultures they 
associate with their communities in their own civic lives and the 
civic lives of the other members of their communities. 


Patriotism 


Love for or devotion to one’s country (or countries), which can lead 
to a willingness to act in support of one’s country (or countries). 


Nationalism 


The politicization of patriotism into principles or programs based on 
the premise that national identity holds precedence over other social 
and political principles. 


Civic and citizenship 


Individuals’ central ethical and moral beliefs about their civic 


values 


communities and their roles as citizens within their communities. 



Civic and citizenship affective-behavioral domains 

The items measuring the affective-behavioral domains do not require correct or incorrect 
responses. They are typically measured through use of a Likert-type item format in the student 
questionnaire. 

Affective-behavioral Domain 1: Value beliefs 

Value beliefs can be defined as beliefs about the worth of concepts, institutions, people, and/ 
or ideas. Value beliefs are different from attitudes insofar as they are more constant over time, 
deeply rooted, and representative of broader and more fundamental beliefs. Value beliefs help 
individuals resolve contradictions, and they form the basis of how we see ourselves and others. 
Value systems are sets of value beliefs that individuals adopt and that, in turn, influence both 
attitudes and behavior. 4 

Value beliefs relevant in the context of civics and citizenship include beliefs about fundamental 
concepts or ideas (freedom, equity, social cohesion). Two types of value beliefs are measured in 
ICCS: 

• Students’ beliefs in democratic values 

• Students' beliefs in citizenship values. 

Students’ beliefs in democratic values 

This construct refers to student beliefs about democracy and relates mainly to Content 
Domain 2 (civic principles). In the IEA CIVED survey, students were asked to rate a number of 
characteristics of society as either “good or bad for democracy.” Contrary to expectations, the 
results did not show similar patterns of conceptual dimensions along which students rate these 
items. However, several items represented a factor related to a “rule of law” model of democracy 
that was consistent across countries (see Torney-Purta et al., 2001). Instead of asking about 
positive or negative consequences for democracy, the ICCS assessment framework includes a set 
of nine items that seek out the extent of student agreement as to what a society should be like. 
The items, adapted from a subset of those included in CIVED, reflect students’ endorsement of 
basic democratic values. In addition, three items reflect students’ beliefs about what should be 
done in response to groups that pose threats to national security. 

4 Rokeach (1973, p. 5) gives the following definitions: “A value is an enduring belief that a specific mode of conduct or end- 
state of existence is personally or socially preferable to an opposite or converse mode of conduct or end-state of existence. 
A value system is an enduring organization of beliefs concerning desirable modes of conduct or end-states of existence 
along a continuum of relative importance.” 
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Students’ beliefs in citizenship values 

This construct refers to student beliefs regarding “good citizenship" and relates mainly to 
Content Domain 1 (civic society and systems). Items asking about the importance of certain 
behaviors for “good citizenship” were included in the first IEA study on civic education in 
1971 (Torney et al, 1975). In CIVED, a set of 15 items asked students to rate the importance 
of certain behaviors relative to being a good citizen (see Torney-Purta et al., 2001, p. 77f). Two 
sub-scales on conventional and social-movement-related citizenship were reported (see Schulz, 
2004). Kennedy (2006) distinguishes active (conventional and social-movement-related) from 
passive citizenship elements (national identity, patriotism, and loyalty). ICCS includes 12 items 
on good citizenship behavior, most of which are similar to those used in CIVED. 

Affective-behavioral Domain 2: Attitudes 

Attitudes can be defined as states of mind or feelings about ideas, persons, objects, events, 
situations, and/ or relationships. In contrast to value beliefs, attitudes are narrower in nature, 
can change over time, and are less deeply rooted. It is also possible for individuals to harbor 
contradictory attitudes at the same time. 

The different types of attitudes relevant with respect to civics and citizenship include: 

• Students' self-cognitions related to civics and citizenship 

• Students' attitudes toward rights and responsibilities 

• Students' attitudes toward institutions. 

Students’ self-cognitions related to civics and citizenship 

Interest in political events and social issues: This construct reflects students’ motivation relative 
to politics and relates to Content Domain 4 (civic identities). The first IEA Civic Education 
Study in 1971 included measures of interest in public affairs television, which turned out to 
be a positive predictor for civic knowledge and participation (Torney et al., 1975). An item 
on political interest was used in the CIVED survey (Torney-Purta et al., 2001). Similar to 
earlier findings, CIVED results also showed interest in politics as a positive predictor of civic 
knowledge and likelihood to vote (Amadeo et al., 2002). ICCS uses a list of items covering 
students’ interest in a broader range of six different political and social issues. An additional 
item, optional for European countries and referring to interest in European politics, is also 
included. 

Self-concept regarding political participation (political internal efficacy): This construct refers to 
students’ self-concept regarding political involvement and relates to Content Domain 3 (civic 
participation). The broader concept of political efficacy has played a prominent role in studies 
on political behavior and political socialization. Political efficacy is the “feeling that political 
and social change is possible and that the individual citizen can play a part in bringing about 
this change” (Campbell, Gurin, & Miller, 1954, p. 187). Analyses of items typically used to 
measure political efficacy reveal a two-dimensional structure of political efficacy, where internal 
efficacy can be defined as individuals’ confidence in their ability to understand politics and to act 
politically, whereas external efficacy constitutes individuals’ beliefs in the responsiveness of the 
political system (see Balch, 1974; Converse, 1972). 

The CIVED survey used three items measuring internal political efficacy, which was positively 
associated with indicators of civic engagement (see Schulz, 2005). In ICCS, the three CIVED 
items are complemented with three additional items. 

Citizenship self-effcacy: This construct reflects students’ self-confidence in active citizenship 
behavior and relates primarily to Content Domain 3 (civic participation). Individuals’ 
“judgements of their capabilities to organise and execute courses of action required to attain 
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designated types of performances” (Bandura, 1986, p. 391) are deemed to have a strong 
influence on individual choices, efforts, perseverance, and emotions related to the tasks. The 
concept of self-efficacy constitutes an important element of Bandura’s (1993) social cognitive 
theory xabout the learning process, which advocates that learners direct their own learning. 

The distinction between self-concept regarding political participation (political internal efficacy) 
and citizenship self-efficacy is as follows: whereas internal political efficacy considers global 
statements regarding students’ general capacity to act politically, citizenship self-efficacy 
considers students’ self-confidence to undertake specific tasks in the area of civic participation. 
ICCS includes seven items reflecting different participation activities. 

Students’ attitudes toward rights and responsibilities 

The following constructs reflect students’ attitudes toward rights and responsibilities and are 
relevant with regard to civics and citizenship. 

Attitudes toward gender rights: This construct reflects student beliefs about rights for different 
gender groups in society. It relates to Content Domain 2 (civic principles). The first IEA Civic 
Education Study in 1971 included four items measuring support for women’s political rights. 
The CIVED survey used a set of six items to capture students’ attitudes toward women’s 
political rights (Torney-Purta et al., 2001). ICCS includes seven items on gender rights, some 
of them identical with or similar to those used in CIVED. 

Attitudes toward the rights of ethnic /racial groups: This construct reflects students’ beliefs about 
rights for different ethnic/ racial groups in society. It relates primarily to Content Domain 
2 (civic principles) and is measured with five items. Four of these items were present in the 
CIVED survey but the results were not included in the international report (Schulz, 2004). 

Attitudes toward the rights of immigrants: This construct reflects students’ beliefs about rights 
for immigrants and it relates to Content Domain 2 (civic principles). CIVED measured this 
construct with eight items, five of which were included in a scale reflecting attitudes toward 
immigrants (Schulz, 2004; Torney-Purta et al., 2001). ICCS includes a slightly modified 
version of the same five items used for scaling, together with one additional item. 

Students’ attitudes toward institutions 

The following constructs reflect students’ attitudes toward institutions and are deemed 
important for civic and citizenship education. 

Trust in institutions: This construct reflects students’ feelings of trust in a variety of state and civic 
institutions in society, and relates mainly to Content Domain 1 (civic society and systems). The 
first IEA Civic Education Study (1971) included one item on trust in government (Torney et al, 
1975). CIVED used a set of 12 items covering political/civic institutions, media, the United 
Nations, schools, and people in general. ICCS uses a similar range of 1 1 core items in modified 
format together with three optional items on European institutions and state/ provincial 
institutions. 

Confidence in student participation at school: This construct reflects students’ beliefs regarding 
the usefulness of becoming actively involved at school. Adolescents are generally not able to 
vote or stand for office in “adult politics,” but they experiment as students to determine what 
degree of power they have to influence the ways schools are run (Bandura, 1997, p. 491). 
CIVED included seven items asking about students’ perceptions of their influence at school. 
Four of these questions focused on general confidence in school participation (Torney-Purta 
et al., 2001). ICCS uses a set of four (partly modified) CIVED items and one additional item 
reflecting student attitudes toward student participation at school. The construct relates to 
Content Domain 3 (civic participation). 
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Attitudes toward one's nation: This construct reflects students’ attitudes toward abstract concepts 
of nation. One can distinguish different forms of national attachment (symbolic, constructive, 
uncritical patriotism, nationalism). These are different from feelings of national identity (Huddy 
& Khatib, 2007). 

The CIVED survey included 1 2 items reflecting students’ attitudes toward their respective 
countries. Four of these items were used to measure “positive attitudes toward one’s nation” 
(Torney-Purta et al., 2001). ICCS uses a set of eight items (four of them from CIVED) to 
measure students’ attitudes toward the country they live in and to cover students’ symbolic 
patriotism and uncritical patriotism. Two of these items relate to the concept of national 
pride (Huddy & Khatib, 2007) but avoid the use of the term “pride” and focus instead on 
students’ perceptions of satisfaction with different aspects (political system and respect for the 
environment) of their respective countries. The construct relates mainly to Content Domain 4 
(civic identities). 

Attitudes toward religion: Religion is sometimes viewed as an important catalyst of civic 
participation (see Verba, Schlozman, & Brady, 1995). ICCS uses a set of six items to assess 
students’ attitudes toward religion. This construct relates mainly to Content Domain 4 (civic 
identities). The set of items forms part of an international option on religious denomination, 
practices, and attitudes toward religion. 

Affective-behavioral Domain 3: Behavioral intentions 

Behavioral intentions refers to student expectations of future civic action. This affective-behavioral 
domain, assessed in the student perceptions questionnaire, contains items that ask students 
about their intentions toward civic action in the near future or when they are adults. Given the 
age group surveyed in ICCS and the limitations for 14-year-olds to participate as active citizens, 
behavioral intentions are of particular importance when collecting data about active citizenship. 

One important aspect of measuring behavioral intentions in the area of civics and citizenship 
is political participation. This aspect can be defined as “activity that has the intent or effect 
of influencing government action — either directly by affecting the making or implementation 
of public policy or indirectly by influencing the selection of people who make those policies” 
(Verba et al, 1995, p. 38). 

During the 1970s and 1980s, protest behavior as a form of participation became more 
prominent in Western democracies (Barnes et al, 1997). Scholars have distinguished 
“conventional” (voting, running for office) from “unconventional” (social movement) activities 
(grassroots campaigns, protest activities). They have also distinguished, among the latter, legal 
from illegal forms of behavior (Kaase, 1990). 

Verba et al. (1995) identify the following three factors as predictors of political participation: 

(a) resources that enable individuals to participate (time, knowledge); (b) psychological 
engagement (interest, efficacy); and (c) “recruitment networks,” which help to bring individuals 
into politics (these networks include social movements, church groups, and political parties). 

The CIVED survey included 1 2 items measuring expected participation (voting, active, 
conventional, unconventional, protest). ICCS has developed a broader set of items that cover a 
wider range of behavioral intentions related to the following constructs or sets of constructs: 

• Preparedness to participate in forms of civic protest 

• Behavioral intentions regarding future political participation as adults 

• Behavioral intentions regarding future participation in citizenship activities. 
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Preparedness to participate in forms of civic protest 

This set of nine items reflects students’ beliefs about future involvement in protest activities (for 
example, collecting petitions, participating in protest marches, blocking traffic). It relates to 
Content Domain 3 (civic participation). The items relate to two different dimensions of protest 
behavior — legal and illegal. 

Behavioral intentions regarding future political participation as adults 

This set of seven core and two optional items reflects two different constructs (voting-related 
participation, political activities) that were measured in CIVED. The set of items reflects 
students’ beliefs about future political participation as an adult (for example, voting in elections, 
active campaigning, joining a party, becoming a candidate) and relates mainly to Content 
Domain 3 (civic participation). 

Behavioral intentions regarding future participation in citizenship activities 

This construct was also included in the CIVED student questionnaire, and it relates mainly 
to Content Domain 3 (civic participation). It reflects students’ beliefs about their future 
participation in citizenship activities (for example, volunteer work, opinion leadership, writing 
letters to newspapers), and is measured with a set of five items in the ICCS assessment. 

Affective-behavioral Domain 4: Behaviors 

Civic-related behavior is limited for 14-year-old students, and many activities for citizens are 
not available at this age. However, several civic-related behaviors can occur among 14-year- 
olds, and the aim is to capture these through the student questionnaire. 

The IEA CIVED survey used a wide range of participation forms both inside and outside of 
school. Reported student participation in a school council or in a student parliament turned 
out to be a positive predictor of civic knowledge and engagement (Torney-Purta et al., 2001; 
Amadeo et al., 2002). Participation in political youth organizations had a positive effect on 
feelings of political efficacy (Schulz, 2005). From their analysis of NAEP data, Niemi and Junn 
(1998) found that participation in role-playing elections or mock trials related positively to 
civic knowledge. 

The ICCS student questionnaire collects data on students’ involvement in 

• Civic-related activities in the community 

• Civic-related activities at school. 

Involvement in civic-related activities in the community 

Students are asked about current or past participation in organizations such as human-rights 
groups, religious associations, and/ or youth clubs. The underlying construct relates mainly to 
Content Domain 3 (civic participation) but is also a relevant contextual variable, as described in 
the contextual framework. 

Involvement in civic-related activities at school 

The ICCS student questionnaire includes questions about a wide range of civic-related 
participation at school (for example, in school councils/ parliaments, in student debates). The 
underlying construct relates primarily to Content Domain 3 (civic participation) and is also 
relevant as a contextual variable, as described in the contextual framework. 
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Civic and citizenship cognitive domains 

To respond correctly to the ICCS cognitive test items, students need to know the core set 
of civic and citizenship content being assessed. Students also need to be able to apply more 
complex cognitive processing to their civic and citizenship knowledge and to relate their 
knowledge and understandings to real-world civic action. 

The two ICCS cognitive domains comprise the cognitive processes that students are expected 
to demonstrate in the ICCS cognitive test. The data derived from the test items constructed 
to represent the processes in the cognitive domains will be used to construct a global scale 
of civic and citizenship knowledge and understandings of the four content domains. The 
first cognitive domain, knowing, outlines the types of civic and citizenship information that 
students are required to demonstrate knowledge of. The second domain, reasoning and analyzing, 
details the cognitive processes that students require to reach conclusions. These processes are 
broader than the contents of any single piece of knowledge, and include the processes involved 
in understanding complex sets of factors influencing civic actions and in planning for and 
evaluating strategic solutions and outcomes. 

Cognitive Domain 1: Knowing 

Knowing refers to the learned civic and citizenship information that students use when engaging 
in the more complex cognitive tasks that help them make sense of their civic worlds. Students 
are expected to recall or recognize definitions, descriptions, and the key properties of civic and 
citizenship concepts and content, and to illustrate these with examples. Because ICCS is an 
international study, the concrete and abstract concepts students are expected to know in the 
core cognitive assessment are those that can be generalized across societies. There is scope in 
the regional modules (where applicable) to target regionally specific knowledge. 

Processes 



Define Identify statements that define civic and citizenship concepts and content. 

Describe Identify statements that describe the key characteristics of civic and 
citizenship concepts and content. 

Illustrate with Identify examples that support or clarify statements about civic and 
examples citizenship concepts and content. 



Cognitive Domain 2: Reasoning and analyzing 

Reasoning and analyzing refers to the ways in which students use civic and citizenship information 
to reach conclusions that are broader than the contents of any single concept. Reasoning 
extends from the direct application of knowledge and understanding to reach conclusions 
about familiar concrete situations through to the selection and assimilation of knowledge and 
understanding of multiple concepts. These outcomes are then used to reach conclusions about 
complex, multifaceted, unfamiliar, and abstract situations. 
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Processes 



Interpret information 


Identify statements about information presented in textual, graphical, 
and/ or tabular form that make sense of the information within the 
context of a civic and citizenship concept. 


Relate 


Use the key defining aspects of a civic and citizenship concept to 
explain or recognize how an example illustrates a concept. 


Justify 


Use evidence and civic and citizenship concepts to construct or 
recognize a reasoned argument to support a point of view. 


Integrate 


Identify connections between different concepts across themes and 
across civic and citizenship content domains. 


Generalize 


Identify civic and citizenship conceptual principles manifested as 
specific examples and explain how these may apply in other civic 
and citizenship contexts. 


Evaluate 


Identify judgments about the advantages and disadvantages of 
alternative points of view or approaches to civic and citizenship 
concepts and actions. 


Solve problems 


Identify courses of action or thought that can be used to alleviate 
civic and citizenship problems expressed as conflict, tension, and/ or 
unresolved or contested ideas. 


Hypothesize 


Predict and support with evidence the effects and outcomes of civic 
and citizenship policies, strategies, and/ or actions. 


Understand civic 
motivation 


Identify the factors that motivate individuals or groups to engage in 
civic action. 


Understand civic 
change 


Identify and explain the factors and processes that lead to change 
in the substance and structure of civic and citizenship concepts and 
entities. 



Mapping items to domains 

The content domains relate to both cognitive and affective-behavioral domains. Any items 
that measure one of the two cognitive domains can be mapped to any of the four content 
domains. The same is true for items measuring any of the affective-behavioral constructs. Table 
1 shows how items can be placed in different cells and mapped to either cognitive or affective- 
behavioral domains as well as to content domains. 

Cognitive items from both domains ( knowing and reasoning and analyzing ;) and affective- 
behavioral items from two domains ( value beliefs and attitudes ) were developed in the contexts 
of all four content domains. Because these mappings are guided by the compatibility of each 
content domain to the different affective-behavioral and cognitive domains, they do not 
necessarily spread evenly across the content domains. Items developed to measure behavioral 
intentions or actual behaviors are only related to Content Domain 3. 
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Table 1: Relationship between cognitive or affective-behavioral and content domains 





Content Domain 1 : 

Civic society 
and systems 


Content Domain 2: 

Civic principles 


Content Domain 3: 

Civic participation 


Content Domain 4: 

Civic identities 


Cognitive 

Domains 




Knowing 


1 


II 


III 


IV 


Analyzing and 
reasoning 


V 


VI 


VII 


VIII 


Affective- 

behavioral 

Domains 




Value beliefs 


A 


B 


C 


D 


Attitudes 


E 


F 


G 


H 


Behavioral 

Intentions 






1 




Behaviors 






J 





The following examples illustrate the mapping of items to domains: 

• A cognitive item that measures student knowledge about the role of parliament is located 
in cell I (Cognitive Domain: Knowing; Content Domain 1: Civic society and systems). 

• A cognitive item measuring student ability to identify the underlying reason for a civic 
protest is found in cell VII (Cognitive Domain: Analyzing and reasoning; Content Domain 
3: Civic participation). 

• An affective-behavioral item asking about students’ agreement that everyone should have 
the right to express his or her opinions freely appears in cell B (value belief related to 
Content Domain 2: Civic principles). 

• An affective-behavioral item asking about students’ trust in parliament is located in cell E 
(attitude related to Content Domain 1: Civic society and systems). 

• An affective-behavioral item asking about students’ expectation to participate in a peaceful 
protest march is located in cell I (behavioral intention related to Content Domain 3: Civic 
participation). 

• An affective-behavioral item asking about students’ past voting in a school election 
appears in cell J (behavior related to Content Domain 3: Civic participation). 
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Contextual Framework 



Classification of contextual factors 

A study of civic-related learning outcomes and indicators of civic engagement needs to 
be set in the context of the different factors influencing them. Young people develop their 
understandings about their roles as citizens in contemporary societies through a number of 
activities and experiences that take place within the contexts of home, school, classrooms, and 
the wider community. 

It is therefore important to recognize that young people’s knowledge, competencies, 
dispositions, and self-beliefs are influenced by variables that can be located at different levels 
in a multi-level structure (see a similar conceptual view in Scheerens, 1990). The individual 
student is located within overlapping contexts of school and home. Both contexts form part 
of the local community that, in turn, is embedded in the wider sub-national, national, and 
international contexts. The contextual framework for ICCS distinguishes the following levels: 

• Context of the wider community: This level comprises the wider context within which schools 
and home environments work. Factors can be found at local, regional, and national levels. 
For some countries, the supra-national level might also be relevant as, for example, in 
member countries of the European Union. 

• Context of schools and classrooms: This level comprises factors related to the instruction 
students receive, the school culture, and the general school environment. 5 

• Context of home environments: This level comprises factors related to the home background 
and the social out-of-school environment of the student (for example, peer-group 
activities). 

• Context of the individual: This level includes the individual characteristics of the student. 

Another important distinction can be made by grouping contextual factors according to those 
related to either antecedents or processes: 

• Antecedents are those factors that affect how student learning and acquisition of civic- 
related understandings and perceptions take place. Note that these factors are level- 
specific and may be influenced by antecedents or processes at a higher level. For example, 
civic-related training of teachers may be affected by historical factors and/ or policies 
implemented at the national level. 

• Processes are those factors related to civic-related learning and the acquisition of 
understandings, competencies, and dispositions. They are constrained by antecedents and 
influenced by factors relating to the higher levels of the multi-level structure. 

Antecedents and processes are factors that shape the outcomes at the level of the individual 
student. Learning outcomes related to civic and citizenship education at the student level also 
can be viewed as aggregates at higher levels (school, country) where they can affect factors 
related to process. For example, higher levels of civic understanding and engagement among 
students can influence the way schools teach civic and citizenship education. 

Figure 3 illustrates which contextual factors might influence the learning outcomes of civic 
and citizenship education. The (double-headed) arrow between processes and outcomes signals 
a reciprocal relationship. It is important to emphasize that “feedback” occurs between civic- 
related learning outcomes and processes. For example, students with higher levels of civic 
knowledge and engagement are those students most likely to participate in activities (at school, 
at home, and within the community) that promote these outcomes. 



5 Because of the sampling design for ICCS, school level and classroom level cannot be disentangled. Generally, only one 
classroom is selected within each sampled school. 
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The (single-headed) arrow between antecedents and processes describes the relationship 
between these two types of factors at each level as uni-directional. However, higher-level 
processes can influence antecedents, and it is likely that, from a long-term perspective, outcomes 
may affect variables that are antecedents for learning processes. 



Figure 3: Contexts for the development of learning outcomes related to civics and citizenship 



Antecedents 



Processes 



Outcomes 



Wider community 




Wider community 


Educational system 




Educational policies 


History and culture 




Political events 


School/classroom 




School/classroom 


Characteristics 




Instruction 


Composition 




Governance 


Resources 






1 1 

1 1 

Student 




i 1 

Student 

1 1 


Characteristics 


► 


i Socialization and 


1 1 
1 1 
1 1 




learning 

i i 


Home environment 




Home environment 


Family background 




Communication 


Social group 




Activities 



Indicators related to 
Civic society and systems 
Civic principles 
Civic participation 
Civic identities 



This general contextual framework for ICCS makes it possible to map variables for which data 
are collected on a three-by-four grid, with antecedents, processes, and outcomes as columns, 
and the levels of nation/ community, school/ classroom, student, and home environment as 
rows. Although the last column for outcomes is not split into levels, it is important to recognize 
that, for the analysis, aggregates can also be used at country and school/ classroom levels. 6 

Table 2 maps examples of potential variables (or groups of variables) collected with different 
ICCS instruments to each cell in this grid. Variables related to the context of nation/ community 
are collected primarily through the national context survey and other possible data sources. 
Variables related to the context of schools and classrooms are collected through the school 
and teacher questionnaires. The student background questionnaire provides information on 
antecedents of the individual student and the home environment. It also provides information 
about process-related variables (for example, learning activities). The student test and the 
student perceptions questionnaire collect data on outcomes. In addition, the student background 
questionnaire includes questions regarding student participation in civic-related activities, which 
are also used as indicators of active citizenship related to Content Domain 3 (civic participation). 

Some potential variables that can be measured at one level pertaining to another level are 
not included in the mapping. Student observations of learning practices in the classroom 
can be aggregated and used as classroom or school variables. Student, school, and teacher 
questionnaires might also provide civic-related information about the context of the local 
community. 

6 Note that similar conceptualizations have been used for the planning of other international studies (see, for example, 
Harvey-Beavis, 2002; OECD, 2005; Travers, Garden, & Rosier, 1989; Travers & Westbury, 1989). 
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Table 2: Mapping of variables to contextual framework (examples) 



Level of ... 


Antecedents 


Processes 


Outcomes 


National and other 
communities 


NCS & other sources: 

Democratic history 
Structure of education 


NCS & other sources: 
Intended curriculum 
Political developments 




School/classroom 


ScQ & TQ: 

School characteristics 
Resources 


ScQ & TQ: 

Implemented curriculum 
Policies and practices 


StT & StQ: 

Test results 
Student perceptions 
Student behaviors 


Student 


StQ: 

Gender 

Age 


StQ: 

Learning activities 
Practiced engagement 


Home environment 


StQ: 

Parent SES 
Ethnicity 
Language 
Country of birth 


StQ: 

Communication 
Peer-group activities 





Note: NCS = national context survey; ScQ = school questionnaire; TQ = teacher questionnaire; StQ = student 
questionnaire; StT = student test; SES = socioeconomic status. 



Contextual levels and variables 

The context of the wider community 

The context of the wider community consists of different levels. The first is the local 
community in which are sited students’ schools and home environments. The second is the 
broader realm of regional, national, and possibly supra-national contexts within which students’ 
schools and homes are embedded. The most relevant levels for ICCS are the national and the 
community contexts. 

National context survey data on the context of the education system 

The ways students develop civic-related dispositions and competencies and acquire 
understandings with regard to their role as citizens are strongly influenced by variables found 
at the country level. Historical background, the political system, the structure of education, and 
the curriculum need to be taken into account when interpreting results from an international 
assessment of civic and citizenship education. 

The national context survey is designed to systematically collect relevant data on the structure 
of the education system, education policy and civic and citizenship education, teacher 
qualifications for civic and citizenship education, and the extent of current debates and reforms 
in this area. The survey also collects process data at the national level regarding assessment of 
and quality assurance in civic and citizenship education and in school curriculum approaches. 

Data from the national context survey are useful for comparing profiles of civic and citizenship 
education in participating countries. In addition, the survey provides data on contextual 
factors that help inform analysis of differences among countries in student knowledge of and 
engagement in civic and citizenship education. 

The structure of the education system: Although a number of global trends in education have 
led to similarities in policies and structures (Benavot et al., 1991), the differences between 
education systems continue to have a considerable effect on the outcomes of education (Baker 
& LeTendre, 2005). 



32 



ICCS ASSESSMENT FRAMEWORK 



To capture these basic differences, the national context survey collects data on length of 
schooling, structure of school education (study programs, public/private management), and 
autonomy of educational providers. 

Education policy and civic and citizenship education: Results from the IEA CIVED survey (Torney- 
Purta et al., 1999) showed that the status of and priority given to civic and citizenship 
education were generally low across countries. Even though civic goals were deemed important, 
their place in the curricula was often not well defined, with civic education typically integrated 
into different subjects. This situation also highlighted the fact that explicit civic and citizenship 
education rarely begins before students reach age 14. 

The national context survey therefore collects data on the definition of, and the priority given 
to, civic and citizenship education in the educational policy and provision of each country, the 
place of civic and citizenship education in educational reforms, and the main aims and goals 
of civic and citizenship education. The survey also asks about the inclusion of specific contexts 
with regard to whole-school approaches, school curriculum approaches, links to the wider 
community in the national or official definition of civic and citizenship education, and the 
influence of different institutions or groups on decisions about the goals and aims of civic and 
citizenship education. 

Approaches to civic and citizenship education: One important aspect of the study is its investigation 
of the extent to which schools in different countries provide support for civic and citizenship 
education through school culture or ethos, democratic school governance, and the establishment 
of links with the wider community (Birzea et al., 2004). Educational policies, for example, may 
include recommendations with regard to the establishment of democratic school practices. 

The national context survey provides country-level data on mandatory or non-mandatory 
recommendations as well as pilot projects or programs concerned with school governance, 
school culture, student participation, parental involvement, and school-community links. It also 
asks these official sources the extent to which they see these aspects as contributing to civic and 
citizenship education. 

Civic and citizenship education and school curriculum approaches: Countries take different approaches 
to the implementation of civic and citizenship education in their curricula (Cox et al., 2005; 
Eurydice, 2005). Some education systems include civic and citizenship education in the 
national curriculum as a compulsory or optional (stand-alone) subject whereas others include it 
as an integral part of other subjects. An alternative approach to civic and citizenship education 
is to implement it as a cross-curricular theme. 

The national context survey gathers data on the inclusion of civic and citizenship education (as 
a separate subject, or integrated into different subjects, or as a cross-curricular approach) in the 
formal curriculum at different stages of schooling and in different study programs. The survey 
also captures the names of specific curriculum subjects and whether they are compulsory or 
optional in each study program. 

Because the ICCS surveys students at a specific target grade in lower secondary programs 
(typically Grade 8), the national context survey includes specific questions relative to this 
grade. The questions concern the common or differentiated curriculum, the main goals of civic 
and citizenship education at this grade, and the extent to which these goals are influenced by 
different institutions and groups. The survey also gathers data on the emphasis the curriculum 
places on teaching different processes (for example, knowing basic facts or understanding key 
concepts) and student learning of specific topics. It furthermore asks about the use of different 
methods for implementing the curriculum (for example, through instructional or pedagogical 
guides) and the amount of instructional time given to civic and citizenship education and the 
amount and type of information given to parents about this area of education. 
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Teachers and civic and citizenship education: The teacher survey undertaken as part of the 
CIVED survey showed considerable diversity in the subject-matter background, professional 
development, and work experience of teachers involved in civic and citizenship education 
(Losito & Mintrop, 2001). With regard to teacher training in this field, research showed a rather 
limited and inconsistent approach to in-service training and professional development (Birzea et 
al., 2004; Eurydice, 2005). 

To assess the variety of different approaches to teacher education in the field at the level of 
education systems, the national context survey gathers general data about the requirements 
for becoming a teacher and about licensing or certification procedures for teachers. More 
specifically, the survey gathers data about the background of civic and citizenship teachers. It 
provides information on the extent to which civic and citizenship education is part of pre- 
service or initial teacher education, on the availability of in-service or continuing professional 
development education for civic and citizenship education, on the providers of these activities, 
and on expectations relating to how teachers learn about changes in the curriculum. 

Civic and citizenship education and assessment and quality assurance: Comparisons of assessment 
and quality assurance for civic and citizenship education are difficult and complex due to the 
diversity of approaches to teaching this subject area across countries. In particular, research in 
Europe shows that, in most countries, and compared to other subject areas, monitoring and 
quality assurance in civic and citizenship education are often unconnected and carried out on a 
smaller scale (Birzea et al., 2004). However, over the last decade, some countries have started to 
implement nationwide assessments, such as CIVED (Torney-Purta et al., 2001). 

The national context survey includes questions about the assessment of student knowledge 
in civic and citizenship education at the lower secondary level, how implementation of this 
subject area is evaluated, and how parents are informed about current approaches to this field of 
learning. 

Current debates and reforms: The last decade has witnessed numerous examples of educational 
reforms in many countries, with the overall aim of improving educational provision and 
outcomes, including those concerning civic and citizenship education. Many of these 
educational reforms were implemented in response to the challenges of learning and living 
in modern societies and because of changes in respective political systems (Cox et al., 2005; 
Torney-Purta et al., 1999). 

The national context survey assesses whether civic and citizenship education is a focus of 
current debates in the country, the nature of such debates, and the general level of interest 
in this area. It also gathers information on current revisions of the curriculum for students at 
the target grade and any revisions of school approaches to civic and citizenship education. It 
additionally collects information about possible educational reforms that may affect the way 
this subject area is taught in schools. 

Data from other sources: Data from official statistics will provide complementary context data at 
the level of countries regarding the structure of the education system, the nature of the political 
system, and the economic and social context of the society. 
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School questionnaire data on the context of the local community 

The community characteristics in which schools and homes are situated vary in their economic, 
cultural, and social resources, and in their organizational features. Inclusive communities 
that value community relations and facilitate active citizen engagement, especially if they are 
well resourced, can offer much to schools and individuals in terms of civic and citizenship 
opportunities for partnerships and involvement. The capacity and the interest of a community 
to engage with its young people can have a strong bearing on young people’s civic and 
citizenship knowledge, dispositions, and competencies in relation to their roles as citizens. 

ICCS uses the school questionnaire to gather data on the contexts and characteristics of the 
local community. Variables pertaining to the community level include urbanization (antecedent), 
resources for citizenship learning in the local area (antecedent), and the existence of civic- 
related activities to promote civic engagement in the context of the local community (process). 

Urbanization: ICCS collects data about the size of the community in which the school is located 
in order to understand if, and to what extent, the level of urbanization (measured in terms of 
number of inhabitants) influences the quantity and the quality of the resources available for 
both schools and students at the community level. 

Resources for citizenship learning in the local area: Differences in quantity and quality of resources 
for citizenship learning available in the local area may have a dual effect. On the one hand, they 
may favor the organization of community-oriented projects (such as environmental education 
projects) and student participation in projects requiring the development of activities involving 
the community, both of which can contribute to developing skills and competencies related to 
civic and citizenship education. On the other hand, community participation in the life of the 
school and in its various levels can be a factor of greater openness and democratization of the 
school itself. 

The model adopted in CIVED recognized the importance of students’ daily lives in their social, 
civic, and political contexts (Torney-Purta et al., 2001). The links between the school and its 
community represent an opportunity for motivating student participation in activities related to 
civic and citizenship education and for offering them real opportunities for exercising the skills 
and competencies necessary for a conscious democratic civic engagement. 

Furthermore, the level of resources may influence the possibilities for the provision of local 
support to schools. Local support can be very important for effective school improvement 
(Reezigt & Creemers, 2005). National support sets the background for improvement whereas 
local support can influence the efforts of schools more directly. Local support includes the wider 
community of the school, the parents of school students, district officials, school administrators, 
and school boards. 

School interactions with their local communities, and the links that have been established with 
other civic-related and political institutions, can also influence student perceptions of their 
relationship with the wider community and of the different roles they may play in it. 

The ICCS school questionnaire includes a set of items asking principals about cultural and 
social resources existing at the local community level (such as libraries, museums, and theatres, 
as well as playgrounds and sports facilities). 

Existence of community activities to promote civic engagement: The characteristics of the community 
in which the school operates can influence the development of effective civic and citizenship 
education. The presence of associations enabling the active exercising of student civic 
engagement and participation is an important factor of possible continuity or discontinuity in 
the students’ experiences in this field, both inside and outside school. 
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The ICCS school questionnaire asks principals about the opportunities students have to 
participate in citizenship-related activities at the community level. 

Existence of social tensions in the community: Because the school is part of the community within 
which it is located, it may be affected by issues and problems existing at the community level. 
Issues of social tension within the local community can influence students’ social relationships 
and the quality of their social lives and everyday experiences, both inside and outside the 
school. 

The ICCS school questionnaire collects data related to principals’ perceptions of those issues 
and the impact these issues have on school life. 

Teacher questionnaire data on the context of the local community 

The teacher questionnaire collects data on teacher/ student participation in civic-related 
activities in the local community and teachers’ personal participation in groups or organizations 
in the local community. 

Teacher/ student participation in civic-related activities in the local community: The teacher 
questionnaire includes a set of items asking teachers about students’ and their own commitment 
to constructing relationships between the school and its community. Items refer to teachers’ 
participation in projects envisaging co-operation between schools and communities and to 
teachers’ direct participation in activities related to civic and citizenship education. 

Two different constructs are investigated: teacher/ student participation in civic-related activities 
organized by the school in the local community and teachers’ personal participation in 
citizenship activities in the local community. 

Similar items are included in the school and in the student questionnaires. The teachers’ answers 
on student engagement in activities related to civic and citizenship education outside the school 
will be analyzed in relation to the principals’ and the students’ answers in order to compare 
their reports. 

The context of schools and classrooms 

It is important, when conducting a study of civic and citizenship education, to take school 
contexts and characteristics into account because these influence the development of young 
people’s knowledge about civics and citizenship, and their dispositions and competencies in 
relation to their roles as citizens. Predominant amongst these is the school’s general ethos, 
culture, and climate within which the policies concerning both the formal and the informal 
civic and citizenship curriculum develop. 

Students’ learning experiences that contribute to their civic and citizenship understandings 
include classroom organization and management, classroom and cross-curricular activities 
and projects, and the resources, materials, and technologies employed in the teaching and 
assessment processes undertaken. The relationships among students and between teacher and 
students are other important aspects of the school context. These relationships are influenced 
by the school’s decisionmaking processes and the opportunities for participation in formal and 
informal governance processes. 

School questionnaire data on the context of schools and classrooms 

The school questionnaires include items seeking information on important antecedent variables 
at the school level, such as principals’ characteristics and school characteristics and resources. 

It also asks about process-related variables concerning school management, school climate, 
teacher, parent, and student participation at school, and the implementation of civic and 
citizenship education at school. 



36 



ICCS ASSESSMENT FRAMEWORK 



School characteristics and resources: School resources consist of both material and human resources, 
and there is strong debate on the extent to which school resources can contribute to school 
development and improvement (Hanushek, 1994, 1997). Beyond the quality of teachers, there 
is evidence that teacher density — as measured by the students to teacher ratio — is associated 
with some student outcomes (McNeal, 1997). 

The ICCS school questionnaire includes items asking about the demographic characteristics 
of schools (Anderson, Ryan, & Shapiro, 1989). These characteristics are associated with the 
variables usually included in a description of a school. The questionnaire also includes a few 
questions asking for fundamental information, notably the type of school (public/ private), 
students’ enrolment, and number of teachers. 

School management: Schools differ within and across countries with reference to the degree 
of autonomy in defining their own educational policies in terms of their management in 
the narrowest sense (school governance, financing, teaching and non-teaching staff) and 
their organization of curricular and teaching and learning activities (curricular contents, 
cross-curricular activities, choice of textbooks, assessment and evaluation) (Eurydice, 2007). 
Individual schools’ degree of autonomy is a factor affecting the possibility of establishing 
specific courses and activities (both curricular and extra-curricular ones) linked to civic and 
citizenship education. A broader degree of autonomy can give greater opportunities for 
the effective participation within democratic school governance of not only teachers and 
students but also of administrative staff, parents, and the community as a whole. The school 
improvement literature shows that at least some autonomy favors the success of improvement 
efforts (Reezigt & Creemers, 2005). 

The questions included in the school questionnaire investigate the level of schools’ autonomy 
in management and in educational planning. Opportunity to organize specific courses, 
projects, curricular and extra-curricular activities, to choose textbooks, and to define criteria 
and procedures for assessment, evaluation, and self-evaluation all contribute to characterizing 
the school educational plan as one that is more or less consistent with the development of 
effective civic and citizenship education. Furthermore, the existence of national legislation and 
regulations and of standards concerning the results that students should achieve, can act, within 
different school systems, as a resource or a constraint for the development of activities related to 
civic and citizenship education. 

The principals’ answers to the questionnaire items also give insight into the extent to which a 
school has autonomy to determine its own educational planning and educational activities and 
how these relate to the construction of an open and democratic school culture. These responses 
furthermore show the extent to which the organization of decisionmaking processes influences 
the participation of teachers, parents, and students in the running of the school. 

Teacher, parent, and student participation at school: Participative governance practices help 
characterize schools as democratic learning environments. Allowing for the participation of 
teachers and parents assists each school to understand the variety of student learning needs 
and to secure teachers’ and parents’ commitment to supporting school educational activities 
(Ranson, Farrell, Peim, & Smith, 2005). 

Students’ participation in the running of the school helps to build a democratic school 
environment and to give students opportunity to develop skills and attitudes related to civic 
and citizenship education. CIVED showed that students’ participation in school councils and 
student parliaments related positively to students’ civic knowledge and engagement (Losito & 
D’Apice, 2003; Torney-Purta et al., 2001). 
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The ICCS school questionnaire includes a set of items asking about students’ participation in 
class/ school representative elections, as well as about teachers’ and parents’ participation in the 
running of the school. 

School climate: This construct refers to “the shared beliefs — the relations between individuals 
and groups in the organization, the physical surroundings, and the characteristics of individuals 
and groups participating in the organization” (Van Houtte, 2005, p. 85). In a civic and 
citizenship education context, school climate can be referred to as the “impressions, beliefs, 
and expectations held by members of the school community about their school as a learning 
environment, their associated behavior, and the symbols and institutions that represent the 
patterned expressions of the behavior” (Homana, Barber, & Torney-Purta, 2005, p. 3). 

A variety of learning situations can affect civic and citizenship education at schools. These 
include leadership and management, everyday activities within the school, and the quality of 
relationships inside the school itself and between the school and the outside community. The 
students’ daily experience in school is a factor that strongly influences students’ perception 
of school as a democratic environment. The possibility of establishing and experiencing 
relationships and behaviors based on openness, mutual respect, and respect for diversity, as well 
as the possibility of giving and asserting one’s own opinion and points of view, allow students 
to practice a democratic lifestyle, to begin exercising their own autonomy, and to develop a 
sense of self-efficacy. 

School climate also relates to the school culture and ethos that contribute to define the school 
as a social organization as well to distinguish each individual school from others (Stoll, 1999). 
School culture refers to patterns of meaning that include norms, beliefs, and traditions shared 
by the members of the school community and that contribute to shaping their thinking and the 
way they act (Stolp, 1994). 

School climate and culture contribute to the development among students, teachers, and non- 
teaching staff of a sense of belonging to the school, thereby enhancing the commitment and 
motivation that these groups have toward improving school educational activities. School staff 
must feel motivated and committed to developing school activities if these are to be successful 
(Reezigt & Creemers, 2005). 

The ICCS school questionnaire includes a set of items asking about the extent to which 
teachers, students, and non-teaching staff numbers feel a sense of belonging to the school. 

Implementation of civic and citizenship education: The CIVED teacher questionnaire included a 
set of items relating to the implementation of civic education at the school level. The ICCS 
school questionnaire includes a set of items that focus on how civic and citizenship education 
is delivered at the school level, the principals’ perceptions of the importance of the aims of 
this area of education, and how specific responsibilities for civic and citizenship education are 
assigned within the school. 

Data from the school instrument are used to look at the actual implementation of civic 
and citizenship education at the level of schools in order to compare the development and 
implementation of this area of education at the national level. The data also provide information 
about the relationships between the intended curriculum and the implemented curriculum. 

Teacher questionnaire data on the context of schools and classrooms 

The teacher questionnaire aims to gather information about teacher characteristics, teachers’ 
participation in school governance, teachers’ views of student influence on school-based 
decisions, teachers’ confidence in teaching methods, teachers’ perception of school climate, 
teaching practices in the classroom, and teachers’ perception of classroom climate and 
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discipline. In addition, one optional section includes questions for teachers of subjects related 
to civic and citizenship education. These questions ask teachers for their views on civic and 
citizenship education at school and on practices used to teach this subject area at school. 

Teacher characteristics: The ICCS teacher questionnaire includes a set of items asking about 
teachers’ demographic variables (gender, age) and work experience (in general and also their 
experience inside their current school). The number of years of teaching inside the current 
school and the holding of specific positions and responsibilities are factors that may influence 
how teachers consider their own relationship with the school, their sense of belonging to it, 
and the extent to which they are willing to take an active part in their school community. These 
factors are therefore ones that contribute to the openness of the school climate. Variables related 
to the teachers’ work experience are subject/ s taught in the target grade, years of teaching, 
years of teaching in the (current) school, and specific positions/ responsibilities within the 
school (head teacher, coordinator of subject areas, department coordinator). 

Teachers’ participation in school governance: The ICCS teacher questionnaire includes a set of 
items asking about teachers’ participation in school governance. The items refer to teacher 
participation in school representative bodies, to their willingness to take on responsibilities 
besides those of teaching, and to their participation in drafting the school education plans. 

The questions also ask the teachers about their participation in civic- and citizenship-related 
activities carried out by the school in the community, and their personal engagement (that is, 
beyond their teaching) in activities of this type. Teacher participation can be seen as a measure 
of both the degree of openness of the school management and of teachers’ commitment toward 
and sense of belonging to their respective schools. 

Teachers’ confidence in teaching methods: The use of teaching and learning methods and classroom 
management procedures that are primarily learner-centered may contribute to the creation of an 
open and democratic classroom climate favorable to acquisition of the skills and competencies 
necessary for active participation and for dealing with situations necessitating problem-solving 
and conflict resolution. Teachers’ confidence in using particular methods and procedures relates 
to both their professional experience and their learning opportunities during their initial 
and in-service training. Indications in this regard also emerged in CIVED. When asked about 
training needs, many of the teachers who participated in this survey expressed their preference 
for training in content areas (Losito & Mintrop, 2001). However, teachers of some countries 
indicated, as important, pedagogical training related to leading classroom discussions and 
fostering an open classroom climate for discussion. 

Because the ICCS teacher questionnaire is addressed to teachers of all subjects, the items 
concerning this construct focus mainly on teachers’ general pedagogical knowledge (Shulman, 
1987). The teachers’ ability to take on a range of managerial responsibilities within and outside 
the classroom (“managerial competence”) as well as their recognition of and “commitment to 
the acknowledgement of the dignity of others [that is, ‘empathy’]” are identified as two of the 
dimensions that contribute to defining quality in teaching (OECD, 1994, p. 35). 

Teachers’ perception of school climate: The ICCS teacher questionnaire includes a set of items asking 
about school climate. The items refer to the school as a democratic learning environment and to 
the contribution of teachers in bringing about a democratic ethos inside the school. 

The teachers’ answers are analyzed in relation to the answers given by the school principals 
in the school questionnaire, as well as to students’ answers to a similar question in the student 
questionnaire, in order to compare the different perspectives. 
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Teaching practices in the classroom: The ICCS teacher questionnaire includes items asking about 
the use of different teaching approaches, including the use of assessment. The use of teaching 
methods that focus on individual students and that favor students’ active participation in 
learning activities contributes to the development of a classroom climate that is open and 
favors the acquisition and exercising of skills and competencies related to civic and citizenship 
education. 

Teachers’ perception of classroom climate and discipline: Classroom climate is a general concept, 
the definitions of which focus mainly on the level of co-operation in teaching and learning 
activities, fairness of grading, and social support. Democratic classroom climate focuses mainly 
on the implementation of democratic and liberal values in the classroom (Ehman, 1980; 

Hahn, 1999). A democratic classroom climate may help students understand the advantages 
of democratic values and practices and may have a positive effect on their active assimilation 
(Perliger, Canetti-Nisim, & Pedahzur, 2006). 

CIVED results highlighted the importance of classroom climate in civic and citizenship 
education (Tornet-Purta et al., 2001). With respect to other variables, classroom climate 
seemed to be one of the factors more directly correlated to student performance and to student 
willingness to engage in civic-related activities. In further analysis, “open classroom climate” was 
used as a predictor of the expectation of participating as an informed voter and of expectations 
of community participation (Torney-Purta & Barber, 2004). 

The construction of an open classroom climate presents a challenge for policy development and 
practice. Students who had high scores on this scale agreed that “students feel free to disagree 
openly with their teachers about political and social issues” and also that “teachers encourage 
the discussion of political social issues about which people have different opinions” (Torney- 
Purta & Barber, 2004, p. 17). Although many teachers across countries agree in principle that 
such discussions are valuable, not every teacher is sufficiently skilled to guide such a discussion. 

The ICCS teacher questionnaire includes a set of items asking teachers about their perception 
of classroom climate and about students’ participation in classroom activities and class 
discussion. 

Teachers’ views on civic and citizenship education: The ICCS teacher questionnaire includes a set of 
items asking teachers how they conceptualize civic and citizenship education, what they see as 
the objectives of this form of education, and how this subject area is delivered in their schools. 
Two constructs are assessed: teachers’ perception of the aims of civic and citizenship education 
at school; and teachers’ opinion on which people should be responsible for civic and citizenship 
education at school. 

The constructs relate to which, in the teachers’ view, are the most important aims of civic 
and citizenship education (development of knowledge and skill, development of a sense of 
responsibility toward the environment, one’s own opinions and social cohesion, development of 
active participation). The items included in the ICCS teacher questionnaire derive in part from 
the CIVED teacher questionnaire. 

Citizenship education and teaching practices at school: The ICCS teacher questionnaire includes an 
international option on civic and citizenship education at school and on the teaching practices 
adopted for teaching civic and citizenship education. This part of the questionnaire is restricted 
to teachers of civic- and citizenship-education-related subjects. 

Constructs and variables included in the international option relate to the planning of civic 
and citizenship education, teaching and learning activities in civic and citizenship education, 
student assessment in civic and citizenship education, teacher confidence in teaching civic- and 
citizenship-related topics, and possible improvements to civic and citizenship education. 

Most of the items for the international option derive from the CIVED teacher questionnaire. 
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Student questionnaire data on the context of schools and classrooms 

The student questionnaire includes questions regarding the classroom climate for civic and 
citizenship education, students’ views of their influence on decisionmaking at school, and 
students’ perceptions of school climate. 

Classroom climate for civic and citizenship education at school: The CIVED survey included a set of 
items measuring students’ perceptions of what happened in their civic education classes. Six 
items were used to measure an index of open climate for classroom discussion (see Schulz, 

2004) that had earlier been identified as a positive predictor of civic knowledge and students’ 
expectations to vote as an adult (Torney-Purta et al., 2001). The ICCS student questionnaire 
includes a similar instrument that measures students’ perceptions of what happens in their 
classrooms during discussions of political and social issues. 

Perceptions about students’ influence on decisionmaking at school: Some scholars argue that more 
democratic forms of school governance contribute to higher levels of student engagement in 
school (see, for example, Mosher, Kenny, & Garrod, 1994, p. 83). However, a more recent 
Swedish study found evidence that student perceptions of direct influence on school or 
classroom matters are negatively associated with civic knowledge (Almgren, 2006). ICCS 
includes a set of seven items asking about the extent to which students think they have a direct 
influence on different types of school matters. 

Student perceptions of school climate: School climate is widely regarded as an important factor in 
explaining student learning outcomes. Scheerens and Bosker (1997, p. 112 ff) view school 
climate as a synonym for a school culture that manifests a range of variables centered on student 
engagement, student absenteeism, student conduct and behavior, staff motivation, and the 
relationships among students, teachers, and the school itself. Homana et al. (2005) emphasize 
the importance of a positive school climate for engaging students in civic-related learning 
experiences. 

The ICCS student questionnaire includes a set of seven items measuring students’ perceptions 
of school and their perceptions of student-teacher relationships at school. 

The context of the home environment 

The home and family contexts and characteristics that can influence the development of young 
people’s knowledge, competencies, behaviors, attitudes, and beliefs in the context of civics and 
citizenship are considerable. They include peer-group interactions, educational resources in the 
home, culture, religion, values, language use, the relationship status young people have within 
their respective families, parental education, incomes and employment levels, access to different 
kinds of media, the quality of the school-home connections, and the wide range of civic- 
related opportunities out of school the young people can access. 

Research findings often emphasize the role of family background for developing positive 
attitudes toward engagement by and participation of young people (see, for example, Renshon, 
1975). However, the school as an agent competing with the home background has been seen as 
even more influential (see, for example, Hess & Torney, 1967). Nonetheless, there is a general 
consensus that family background is an influential variable in the political development of 
adolescents. The role of socioeconomic background can be seen as influential in providing a 
more stimulating environment and enhancing the educational attainment and future prospects 
of adolescents. These factors, in turn, foster political involvement as an individual resource. 

Studies of political socialization and participation emphasize the importance of the extent 
to which families and individuals can access different forms of capital. Bourdieu (1986) sees 
economic capital as the sources of other forms of capital and distinguishes between human, 
cultural, and social capital. Whereas human capital refers to an individual’s skills, knowledge, 
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and qualifications, cultural capital refers to those “widely shared, high-status cultural signals 
(attitudes, preferences, formal knowledge, behaviours) used for social and cultural exclusion” 
(Lamont & Lareau, 1988, p. 156). Social capital is conceptualized as a societal resource that 
links citizens to one another so that they can achieve goals more effectively (Stolle with Lewis, 
2002 ). 

In his study of institutional performance in Italy, Putnam (1993, p. 185) positioned social 
capital as the “key to making democracy work.” His conceptual view built on Coleman’s (1988) 
concept of social capital that is generated by the relational structure of interactions inside 
and outside the family and which facilitates the success of individuals’ actions and also their 
learning outcomes. 7 According to Putnam (1993), three components of social capital (social 
trust, social norms, and social networks) form a “virtuous cycle” that provides a context for 
successful co-operation and participation in a society. 

Social capital research uses various groups of different factors, including socioeconomic 
status, personal networks, memberships in organizations, interpersonal trust, and personal 
communication (media, discussions). The concept of social capital consequently is often 
criticized for its lack of clarity and the problems it presents in terms of finding suitable 
indicators (Woolcock, 2001). 

Within the context of ICCS, the concept of social capital is viewed as helpful in that it 
describes mechanisms that explain why some students have higher levels of civic knowledge 
and engagement than others. Measures of different aspects of social capital (trust, norms, and 
social interaction) include attitudinal and background variables. Some variables reflecting social 
capital are related to the home environment, in particular interactions with parents, peers, and 
media. Other variables relevant in this context are measures of interpersonal trust and voluntary 
participation in civic-related organizations (see Civic and Citizenship Framework above). 

Variables related to the home environment that are antecedents of student learning and 
development and are measured through the student background questionnaire include (i) 
parental socioeconomic status, (ii) cultural and ethnic background, (iii) parental interest 
in political and social issues, and (iv) family composition. The ICCS student background 
questionnaire also collects data on process-related variables that reflect social interactions 
outside of school (for example, discussing political and social issues with parents and peers and 
accessing media information). 

Parental socioeconomic status: Socioeconomic status (SES) is widely regarded as an important 
explanatory factor that influences learning outcomes in many different and complex ways 
(Saha, 1997). There is general agreement that SES is represented by income, education, and 
occupation (Gottfried, 1985; Hauser, 1994) and that using all three variables is better than 
using only one (White, 1982). However, there is no agreement among researchers on which 
measures should be used in any one analysis (Entwisle & Astone, 1994; Hauser, 1994). 

In international studies, the additional caveats imposed on the validity of background measures 
and the cross-national comparability of family background measures present an ongoing 
challenge for researchers in this area (see Buchmann, 2002). 

The ICCS survey collects three different types of measures through the student background 
questionnaire: 

• Data on parental occupation are collected through open-ended student reports on mother’s 
and father’s jobs and coded according to the ISCO-88 classification (International Labour 



7 Putnam’s view of social capital, however, is narrower and more specific than Coleman’s concept. Putnam saw social 
capital as a collective resource and stated that horizontal interactions tend to foster trust and participation whereas vertical 
relationships lead to distrust and disengagement (Stolle with Lewis, 2002). 
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Organisation, 1990), which in turn is scored using the International Socio-economic Index 
(SEI) of occupational status in order to obtain SES measures (Ganzeboom, de Graaf, & 
Treiman, 1992). 

• Data on parental education are collected through closed questions in which educational 
levels are defined by the ISCED-97 classifications (UNESCO, 2006) and then adapted to 
the national context. 

• Data on home literacy environment are collected through a question about the number of 
books at home. 

Cultural / ethnic background: International studies confirm differences in reading literacy relative 
not only to language and immigrant status (see, for example, Elley, 1992; Stanat & Christensen, 
2006) but also to mathematics literacy (Mullis et al., 2000). Students from immigrant families, 
especially those recently arrived in a country, tend to lack proficiency in the language of 
instruction and to be unfamiliar with the cultural norms of the dominant culture. Also, ethnic 
minorities often have a lower SES, which correlates highly with learning and engagement; there 
is also evidence that immigrant status and language have a unique impact on student literacy 
(Lehmann, 1996). 

ICCS measures cultural and ethnic family background via the following variables: 

• Country of birth (mother, father, and student)-. This information is used to distinguish “native,” 
“first-generation” (parents born abroad but student born in country), and “immigrant” 
(parents and student born abroad) students. 

• Language of use at home (language of test versus other languages). 

• Student self reports on ethnicity (this question is optional for countries). 

Parental interest: There is evidence that young people with parents engaging them in discussions 
about politics and civic issues tend to have higher levels of civic knowledge and engagement 
(see, for example, Lauglo and 0ia, 2006; Richardson, 2003). The ICCS survey asks students to 
what extent their parents are interested in political and social issues. 

Family composition: Family structure represents an important factor of socialization, which may 
affect learning outcomes. Research in the United States, for example, shows that students from 
single-parent families perform less well than those from two-parent households. This finding 
typically has been associated with economic stress and lack of human or social capital in the 
household (McLanahan & Sandefur, 1994; Seltzer, 1994). However, the effects of single-parent 
upbringing on learning outcomes are generally considered to be relatively small (for a review, 
see Marjoribanks, 1997). 

ICCS measures family structure by asking students about the composition of their respective 
households, that is, parents, guardians, siblings, relatives, and/ or other persons. (This question is 
optional for countries.) 

Indicators of social interaction: Analysis of CIVED data showed that frequency of political 
discussions is a positive predictor of both feelings of efficacy and expected participation (see, 
for example, Richardson, 2003). Similar results were found in a comparative study of lower and 
upper secondary students in 15 countries that participated in CIVED (Schulz, 2005). 

One popular explanation for the waning of civil society in the United States is the negative 
effect of television viewing (Putnam, 2000), which leads to decreases in interest, sense of 
efficacy, trust, and participation (see also Gerbner, 1980; Robinson, 1976). However, research 
shows that media use (in particular for information) is usually positively related to political 
participation. Norris (2000), for example, found no conclusive evidence from an extensive 
literature review and findings from a large-scale study for a negative relationship between media 
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use and political participation. CIVED showed that obtaining information through television 
news is a positive predictor of civic knowledge and expected participation in elections (Torney- 
Purta et al., 2001). 

Researchers suggest that religious affiliation may help to foster political and social engagement 
(see, for example, Verba, Schlozman, & Brady, 1995) because religious organizations provide 
networks focused on political recruitment and motivation. However, there is also evidence of 
negative effects of religious affiliation on democratic citizenship, as reflected in lower levels of 
political knowledge and feelings of efficacy among strongly religious people (Scheufele et ah, 
2003). In the case of young people, religious affiliation and participation can be seen as part of 
the home environment and its influence on civic-related learning. 

The following variables reflect students’ interactions in the context of the home environment 
and/ or peer-group: 

• Participation in discussions about political social issues with parents or peers 

• Media information from television, radio, newspapers and/ or the internet 

• Participation in religious services (optional). 

The context of the individual student 

The extent to which each student develops understandings, competencies, and dispositions 
can be influenced by a number of characteristics, some of which link to family background. 
Antecedents at this level, collected through the student questionnaire, include the student 
characteristics of age, gender, and expected educational qualifications. The student questionnaire 
also collects process-related factors, such as leisuretime activities and active civic participation at 
school and in the community. 

Age: Research has found that adolescent students’ civic knowledge and (at least some forms of) 
engagement increase with age (Amadeo et al., 2002; Hess 8c Torney, 1967). However, there is 
also evidence that students’ feelings of trust in the responsiveness of institutions and willingness 
to engage in conventional forms of active political participation decrease as they near the end of 
their secondary schooling (Schulz, 2005). 

In cross-sectional research based on grade sample data, age tends to be negatively correlated 
with student performance in general. This association is particularly evident in countries with 
higher rates of grade repetition. The reason why is that the older students in the class are 
typically those students who are repeating a grade because of low achievement. 

Gender (male, female): The first IEA Civic Education Study in 1971 found considerable gender 
differences relative to cognitive achievement, with males tending to have the higher civic 
knowledge scores (Torney et al., 1975). The IEA CIVED survey, however, presented a different 
picture: whereas in some countries males showed (slightly and not significantly) higher average 
scores, in other countries females were performing better (although only one country reported 
the difference as significant). Interestingly, somewhat higher gender differences in favor of 
males were found in the follow-up study of upper secondary students (Amadeo et al., 2002). 

CIVED also showed that gender differences were usually larger with regard to indicators of 
civic engagement: in most countries, males tended to have higher levels of political interest and 
expected participation. Gender differences were also important with regard to attitudes toward 
immigrants’ and women’s rights (Amadeo et al., 2002; Torney-Purta et al., 2001). 

Expected educational level: In the first two IEA studies on civic education, expected years of 
future education were important predictors of civic knowledge (Amadeo et. al., 2002; Torney 
et al., 1975; Torney-Purta et al., 2001). This variable reflects individual aspirations. However, 
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responses can also be influenced by parent or peer expectations and/ or, in some education 
systems, by limitations brought about by students studying in programs that do not give access 
to university studies. 

A shortcoming of asking about years of further education is that students do not necessarily 
know how long they might take to reach certain qualifications (in particular at the tertiary 
level). A decision was therefore made to include a modified question in the ICCS survey 
that asks students about the educational qualifications they expect to attain. The educational 
qualifications listed in the survey follow the ISCED qualifications (UNESCO, 2006) and are 
adapted to each country’s education system. 

Out-of-school activities: CIVED included a few indicators of student activities outside of school. 
Higher frequencies of students spending time with peers outside their homes were found to 
be a negative predictor of civic knowledge (Torney-Purta et al., 2001). The ICCS student 
questionnaire includes a set of items regarding student behavior outside of school. These items 
include television watching, spending time with peers, and reading for enjoyment. 

Civic participation: The ICCS student questionnaire collects data on students’ involvement in 
civic-related activities in the community and on their involvement in civic-related activities 
at school. These variables are described in detail as behavioral constructs in the Civic and 
Citizenship Framework above. 
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Assessment Design 



The ICCS instruments 

The ICCS instruments collect data relative to outcome as well as contextual variables. Given 
the specific nature of a study on civic and citizenship education, outcome variables are assessed 
through cognitive test material and a student questionnaire. Contextual data that explain 
variation in outcome variables are collected through student, teacher, and school questionnaires 
as well as through the national context survey. 

Table 3 lists the instruments used in ICCS, their approximate administration times, and their 
respondents. The student assessment instrument consists of two parts: (i) an international 
core, including the cognitive test and the student questionnaire; and (ii) an optional regional 
instrument that includes a regional test and a questionnaire. 8 

Three regional modules are available for the countries in Asia, Europe, and Latin America. 
Student-level instruments have been developed for each module. Regional instruments are 
administered in the countries participating in these regional modules, but not until the two 
international core instruments (student test and student questionnaire) have been completed. 



Table 3: ICCS instruments 



Instrument 


Length 


Respondent 


International cognitive test 


45 min. 


Student 


International student questionnaire 


40 min. 


Student 


Regional module instrument 


~30 min. 


Student 


Teacher questionnaire 


~30 min. 


Teacher 


School questionnaire 


~30 min. 


Principal 


National context survey 


30-60 min. 


NRC 



Note: NRC = national research coordinator or designate. 



Both the international student test and the international student questionnaire include sets 
of items that were used in the IEA CIVED survey in 1999. A set of 17 non-released CIVED 
test items will be used to estimate measures of trends for those countries participating in both 
surveys. The student questionnaire assesses a number of constructs that were also measured in 
CIVED through use of identical or similar sets of items. 

The assessment framework has been central to the process of instrument development because 
it provides a theoretical underpinning and describes the areas of assessment. It has guided the 
development of all ICCS instruments and served as a point of reference for the development of 
instruments for the three regional modules. 



8 The Asian regional instrument consists of a questionnaire only. 
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The coverage of framework domains 

Table 4 shows the mapping of cognitive and affective-behavioral domains to content domains. 
It also shows, within each cell, coverage of the cognitive and perceptions items included in the 
international student test and questionnaire. 



Table 4: Coverage of cognitive/ affective-behavioral and content domains in the ICCS student survey 





Content domains 


Total 


Civic society 
and systems 


Civic 

principles 


Civic 

participation 


Civic 

identities 


Cognitive domains 












Knowing 


15 


3 


1 


0 


19 


Analyzing and 
reasoning 


17 


22 


17 


5 


61 


Total 


32 


25 


18 


5 


80 


Affective-behavioral 

domains 












Value beliefs 


12 


12 


0 


0 


24 


Attitudes 


12 


18 


18 


14 


62 


Behavioral Intentions 






21 




21 


Behaviors 






14 




14 


Total 


24 


30 


53 


14 


121 



Note: The table does not include optional student questionnaire items. 



The cognitive items from both domains (knowing and reasoning and analyzing ) and the affective- 
behavioral items from two domains (value beliefs and attitudes) were developed within the 
contexts of all four content domains. However, the items are not spread evenly across the cells 
in the table: most items measuring the cognitive domain knowing relate to the content domain 
civic society and systems. The affective-behavioral items measuring value beliefs relate to two of 
the four content domains only (civic society and systems and civic principles). 

Item types 

The ICCS instruments include a range of different item types in order to assess a diversity of 
cognitive, affective-behavioral, or contextual aspects. 

The cognitive test contains two item types: 

• Multiple-choice (MC): Each item has four response options, one of which is the correct 
response and the other three of which are distracters. 

• Open-ended response (OR): Students are requested to write a short response to an open-ended 
question. The responses are scored by scorers working for the national centers. 

Six of the 80 ICCS test items are open-ended response items. All other items have a multiple- 
choice format. Test questions are typically organized in units within which the content of all 
items refers to a stimulus describing a particular situation or problem and, in a few cases, is 
accompanied by a graphic. Appendix B provides examples of ICCS test questions. 
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The student, teacher, and school questionnaires have the following item types: 

• Likert-type items: For each item, respondents are asked to rate a number of statements, 
typically on a four-point scale. For most of these items, the rating scale ranges from 

(1) strongly agree to (4) strongly disagree. The rating scales for other questions indicate 
frequencies (never, rarely, sometimes, often) or levels of interest, trust, or importance. 

• Multiple-response items: Respondents are asked to indicate the three aspects they view as 
most important. 

• Categorical response items: Respondents are required to choose from two or more response 
categories the response that they consider most appropriate. These questions are primarily 
used for collecting contextual information (for example, on gender, educational level 

of parents, books in the home, subjects taught at school, and public or private school 
management). 

• Open-response items: Respondents are asked to write short responses that are coded by the 
national centers. (These items are used only used for collecting information on parental 
occupation.) 

The ICCS test booklet design 

ICCS uses a rotated design for test administration, making it possible to include more test 
material and thus ensure greater coverage of the assessment framework without increasing the 
testing time for each student. This procedure also enables a sufficient number of score points 
to be generated to provide the basis for comprehensive descriptions of the scale. Rotating the 
clusters throughout the booklets ensures that the different tests are linked and can be scaled 
using IRT (item response theory) methods (Hambleton, Swaminathan, & Rogers, 1991). 

Table 5 shows the booklet design for the ICCS main survey. Cluster C07 contains the CIVED 
link items and that will appear in Booklets 4, 6, and 7. The booklet design is completely 
balanced, which means that each cluster appears in three booklets in three different positions. 



Table 5: Field trial test booklet design 





Position 


Booklet 


A 


B 


C 


1 


C01 


C02 


C04 


2 


CO 2 


C03 


CO 5 


3 


C03 


C04 


C06 


4 


C04 


C05 


C07 


5 


CO 5 


C06 


C01 


6 


CO 6 


C07 


C02 


7 


C07 


C01 


C03 
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Reporting on the contextual scales 

ICCS will report outcomes of civic and citizenship education on a number of international 
scales derived from the student test and the student questionnaire. 

The cognitive test items will be scaled to obtain scores of civic knowledge and understanding. 

The scale will cover student knowledge and understanding with regard to the four content 
domains and the two cognitive domains. Items will be used to describe student knowledge and 
understandings at different levels of student proficiency. 

The student questionnaire includes items that are used to obtain the following scales reflecting 
students’ value beliefs: 

• Students’ support for democratic value beliefs 

• Students' support for the importance of conventional citizenship 

• Students' support for the importance of social-movement related citizenship. 

The student questionnaire includes items that are used to obtain scales reflecting the following 
attitudes: 

• Students' interest in political and social issues 

• Students' civic self-concept 

• Students’ support for equal gender rights and responsibilities 

• Students' support for equal rights for ethnic/ racial groups 

• Students’ support for equal rights for immigrants 

• Students' confidence in school-based participation 

• Students' trust in government-related institutions 

• Students' sense of citizenship self-efficacy 

• Students' attitudes toward their country. 

The student questionnaire includes items that are used to obtain scales reflecting the following 
behavioral intentions: 

• Students' expectations to participate in legal protest activities 

• Students’ expectations to participate in illegal protest activities 

• Students’ expectations to participate in elections 

• Students’ expectations to engage in active political participation 

• Students’ expectations to participate in informal political participation. 

The student questionnaire includes items that are used to obtain scales reflecting the following 
behaviors: 

• Students’ reports of civic participation outside of school 

• Students' reports of civic participation at school. 

Additional scales reflecting contextual factors are derived from the student, teacher, and 
school questionnaires. These scales are designed to describe the learning context for civic and 
citizenship education and to explain variation in learning outcomes. 
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The student questionnaire includes items that are used to obtain scales reflecting the learning 
context of students: 

• Students’ perceptions of student— teacher relationships at school 

• Students’ perceptions of open classroom climate for discussion 

• Students’ perceptions of student influence on decisionmaking at school 

• Students’ reports of discussion of political and social issues with parents and peers. 

The teacher questionnaire includes items that are used to obtain scales reflecting the following 
teacher perceptions of the school environment: 

• Teachers’ personal participation in activities outside school 

• Teachers’ participation in school governance 

• Teachers’ perceptions of student activities in the community 

• Teachers’ perceptions of student influence at school 

• Teachers’ perceptions of social problems at school. 

The teacher questionnaire includes items that are used to obtain scales reflecting the following 
teacher perceptions of school and classroom climate: 

• Teachers’ perceptions of students’ sense of belonging to the school 

• Teachers’ perceptions of student participation in class activities 

• Teachers’ perceptions of student participation in class discussion 

• Teachers’ perceptions of classroom climate. 

The international option included in the teacher questionnaire includes items that are used 
to obtain a scale reflecting teachers’ perceptions of how civic and citizenship education is 
implemented at the school level: 

• Teachers’ reports on the use of traditional teaching methods in civic and citizenship education 

• Teachers’ reports on the use of active teaching and learning in civic and citizenship education. 

The school questionnaire includes items that are used to obtain scales reflecting principals’ 
perceptions of the school environment: 

• Principals’ perceptions of school autonomy in management 

• Principals’ perceptions of school autonomy in educational planning 

• Principals’ perceptions of teacher participation in school governance 

• Principals’ perceptions of student opportunities to participate in community activities 

• Principals’ perceptions of parents’ participation in school life 

• Principals’ perceptions of student influence at school 

• Principals’ perceptions of student behavior at school 

• Principals’ perceptions of teachers’ sense of belonging to the school 

• Principals’ perceptions of students’ sense of belonging to the school 

• Principals' perceptions of non-teaching staff’s sense of belonging to the school 

• Principals' perceptions of social tension in the local community 

• Principals' perceptions of social problems at school. 
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Appendices 



APPENDIX A: INSTITUTIONS AND STAFF 

The international study center and its partner institutions 

The international study center is located at the Australian Council for Educational Research 
(ACER) and serves as the international study center for ICCS. Center staff are responsible for 
the design and implementation of the study in close co-operation with the center’s partner 
institutions NFER (National Foundation for Educational Research, Slough, United Kingdom) 
and LPS (Laboratorio di Pedagogia Sperimentale at the Roma Tre University, Rome, Italy) as 
well as the IEA Data Processing and Research Center (DPC) and the IEA Secretariat. 

Staff at ACER 

John Ainley, project coordinator 
Wolfram Schulz, research director 

Julian Fraillon, coordinator of test development and the Asian regional module 
Naoko Tabata, project researcher 
Tim Friedman, project researcher 

Staff at NFER 

David Kerr, associate research director 
Joana Lopes, senior research officer 

Staff at LPS 

Bruno Losito, associate research director 
Gabriella Agrusti, project researcher 

The International Association for the Evaluation of Educational Achievement (IEA) 

IEA provides overall support in coordinating ICCS. The IEA Secretariat in Amsterdam, the 
Netherlands, is responsible for membership, translation verification, and quality control 
monitoring. The IEA Data Processing and Research Center (DPC) in Hamburg, Germany, is 
mainly responsible for sampling procedures and the processing of ICCS data. 

Staff at the IEA Secretariat 

Hans Wagemaker, executive director 
Barbara Malak, manager membership relations 
Jur Hartenberg, financial manager 

Staff at DPC 

Heiko Sibberns, co-director 
Dirk Hastedt, co-director 
Falk Brese, ICCS coordinator 
Michael Jung, researcher 
Olaf Zuehlke, researcher (sampling) 

Sabine Meinck, researcher (sampling) 
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The ICCS project advisory committee (PAC) 

PAC has, from the beginning of the project, advised the international study center and its 
partner institutions during regular meetings. PAC also reviewed draft versions of the assessment 
framework and the instruments. 

PAC members 

John Ainley (chair), ACER, Australia 

Barbara Malak, IEA Secretariat 

Heiko Sibberns, IEA Technical Expert Group 

John Annette, University of London, United Kingdom 

Leonor Cariola, Ministry of Education, Chile 

Henk Dekker, University of Leiden, the Netherlands 

Bryony Hoskins, Center for Research on Lifelong Learning, European Commission 

Rosario Jaramillo F., Ministry of Education, Colombia 

Judith Torney-Purta, University of Maryland, United States 

Lee Wing-On, Hong Kong Institute of Education, Hong Kong SAR, China 

Christian Monseur, University of Liege, Belgium 

The ICCS sampling referee 

Jean Dumais from Statistics Canada in Ottawa is the sampling referee for the study. He has 
provided invaluable advice on all sampling-related aspects of the study. 

National research coordinators 

The national research coordinators (NRCs) play a crucial role in the development of the project. 
They provided policy- and content-oriented advice on the development of the assessment 
framework and instruments and are responsible for the implementation of ICCS in participating 
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Austria 

Gunther Ogris 

SORA Institute for Social Research and Analysis, Ogris & Hofinger GmbH 
Belgium (Flemish) 

Saskia de Groof 
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Meihui Liu 

Taiwan Normal University, Department of Education 
Colombia 

Margarita Pena 

Instituto Colombiano para el Fomento de la Educacion Superior (ICFES) 
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Czech Republic 

Petr Soukup 
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Jens Brum 

The Danish University of Education, Department of Educational Anthropology 
Dominican Republic 

Julio Leonardo Valeiron Urena 
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England 

Julie Nelson 

National Foundation for Educational Research 

Estonia 

Anu Toots 

Tallinn University 

Finland 

Pekka Kupari 

Institute for Educational Research, University of Jyvaskyla 

Greece 

Georgia Polydorides 

Department of Early Childhood Education 

Guatemala 

Luisa Muller Duran 

SINEIE — Ministerio de Educacion 

Hong Kong SAR, China 
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Hong Kong Institute of Education 
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Balitbang Diknas, Depdiknas 
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Educational Research Centre, St Patrick’s College 

Italy 
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Republic of Korea 

Tae-Jun Kim 

Korean Educational Development Institute (KEDI) 
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Andris Kangro 

Faculty of Education and Psychology, University of Latvia 

Liechtenstein 

Horst Biedermann 

Universitat Freiburg, Padagogisches Institut 
Lithuania 

Zivile Urbiene 

National Examination Center 
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Luxembourg 

Joseph Britz 

Ministere de l’Education nationale 

Malta 

Raymond. Camilleri 

Department of Planning and Development, Education Division 

Netherlands 

M. P. C. van der Werf 

GION, University of Groningen 

New Zealand 

Sharon Cox 

Comparative Education Research Unit, Ministry of Education 

Norway 

Rolf Mikkelsen 

University of Oslo 

Paraguay 

Agustina Solami'a Burgos 

Direction General de Desarrollo Educativo, Ministerio de Education y Cultura 
Poland 

Krzysztof Kosela 

Institute of Sociology, University of Warsaw 

Russia 
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Institution for Education Reforms of the State University Higher School of Economics 
Slovak Republic 

Ervin Stava 
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Marjan Simenc 

University of Ljubljana 

Spain 

Rosario Sanchez 
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Sweden 

Fredrik Lind 

The Swedish National Agency for Education (Skolverket) 
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Thailand 
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APPENDIX B: SAMPLE TEST ITEMS 



Example Item 1 

Example Item 1 is an adaptation of an item used in the ICCS field trial. This item accesses: 

• Content Domain 1: Civic Society and Systems 
o Sub-domain: State institutions 

• Aspect: Legislatures/ parliaments 
o Key concept: Voting/ franchise. 

• Cognitive Domain 1: Knowing 
o Process: Describe 



El 


Which people are allowed to vote in a country's national <parliament/legislature>? 


*□ 


representatives voted into office in national elections 


□ 


judges who sit in the highest court in the country 


□ 


business leaders who pay a fee to be allowed to vote 


□ 


senior members of the national police force 



Note: • = correct answer. 
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Example Items 2 and 3 

Example Items 2 and 3 were presented together as a unit in the ICCS field trial. A unit is one 

or more items that appear in sequence and refer to a common theme or set of information. 

Example item 2 accesses: 

• Content Domain 1: Civic Society and Systems 
o Sub-domain: State institutions 

• Aspect: Government 
o Key concept: Power/ authority 

• Cognitive Domain 2: Reasoning and analyzing 
o Process: Generalize 



The United Nations has election monitors who check whether national elections have taken place fairly. 
The United Nations can only send election monitors to visit a country if the government of that country 
asks them to come. 

<Zedland> asked the United Nations to monitor their national election. After the election, the election 
monitors reported that the election was fair. 



E2 


How does the report that the election was fair help the newly elected government lead 
<Zedland>? 


□ 


People who did not vote for the new government are more likely to change their minds and 
agree with all the decisions the new government makes. 


*□ 


People who did not vote for the new government are more likely to accept the authority of the 
new government to make decisions. 


□ 


People who voted for the new government are more likely to vote for it again in the future. 


□ 


People who voted for the new government are more likely to agree with everything it does. 



Note: ’ = correct answer. 
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Example Item 3 accesses: 

• Content Domain 1: Civic Society and Systems 
o Sub-domain: State institutions 

• Aspect: Government 
o Key concept: Democracy 

• Cognitive Domain 2: Reasoning and analyzing 
o Process: Integrate 



E3 


What message might the United Nations hope to give the world by working with countries to 
monitor their elections? 


□ 


Most countries do not want to conduct fair elections. 


□ 


Countries only need to have fair elections when the election monitors are involved. 


*□ 


Countries have a responsibility to make sure that elections are fair. 


□ 


Fair elections can only be guaranteed if election monitors are present. 



Note: • = correct answer. 



Example Item 4 

Example Item 4 is a CIVED release item and was therefore developed with reference to the 
CIVED conceptual model. Seventeen of the 80 ICCS test items are CIVED-trend (non-release) 
items. Example Item 4 illustrates both the “nature” of the CIVED items and their similarity 
to and congruence with the newly developed ICCS items. The ICCS assessment framework 
incorporates and builds on the CIVED conceptual model. The CIVED non-release trend items 
were included in the ICCS test instrument to facilitate the measurement of student cognitive 
achievement in ICCS on the same metric used in CIVED. It is therefore possible to map the 
CIVED test items (trend and release) onto the ICCS assessment framework. Example Item 4 can 
be mapped to: 

• Content Domain 1: Civic Society and Systems 
o Sub-domain: Civil institutions 

• Aspect: Political parties 
o Key concept: Democracy 

• Cognitive Domain 1: Knowing 
o Process: Describe 



E4 


In democratic countries what is the function of having more than one political party? 


*□ 


To represent different opinions [interests] in the national legislature [e.g. Parliament, Congress] 


□ 


To limit political corruption 


□ 


To prevent political demonstrations 


□ 


To encourage economic competition 



Note: • = correct answer. 
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Example Item 5 

Example Item 5 is one of eight open-ended response items used in the ICCS field trial. The 
scoring guide for Example Item 5 is included on the next page in order to illustrate the 
different categories of credit that were allocated to different conceptual categories of student 
response to the item. Example Item 5 accesses: 

• Content Domain 3: Civic Participation 

o Sub-domain: Community participation 
• Aspect: Volunteering 
o Key concept: Civic involvement. 

• Cognitive Domain 1: Reasoning and analyzing 
o Process: Understand civic motivation 



A local school has a volunteer day. On this day parents volunteer to come to the school and paint the 
classrooms. The parents are not paid for their work. 

<Male Name> is a parent who does not like painting, but he volunteered anyway. 



E5 


Write the best reason to explain why <Male Name> volunteered to help paint the classrooms. 















ICCS ASSESSMENT FRAMEWORK 
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Example Item 5 Scoring Guide 

The example student responses in the scoring guide for Example Item 5 are all “real” responses 
provided by students during the ICCS field trial. 

Full credit 

Code 2: Refers to either or both of the two categories of reason listed below. 

RC 1 . a concern for the common good 

RC2. a desire to participate in the local community. 

RC1. Suggests a concern for the common good as a motivation for volunteering. [Note: A desire to help others or to help the 
school is seen as sufficient for this category.] 

Example responses: 

• He wants to contribute to the school. 

• It’s the right thing to do. 

• Because he’s helping other people, even if he doesn’t want to he would feel better 
knowing he helped them. 

RC2. Suggests a desire to participate in the school/ local community as a motivation for volunteering. 

Example responses: 

• He wants to get involved in his child’s school. 

• He wants to feel part of the local community and the school. 

• So he can socialize and have fun. 

Partial credit 

Code 1 : Suggests only immediate self-interest or benefit as a motivation for volunteering. 

Example responses: 

• He wants to look important/ wants to look good. 

• He wants to learn new skills. 

• To make sure he knows the environment his child is working in. 

No credit 

Code 0: Refers to a generalized personality quality of <Male Name> or provides an irrelevant, 
incoherent response, or repeats the question. 

Example responses: 

• He doesn’t want to refuse, [vague] 

• Because his children go there, [vague] 

• Because he is probably a good man, is generous and it was a chance for him to 
volunteer. 
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This document outlines the framework and assessment design for the 
International Civic and Citizenship Education Study (ICCS) sponsored by the 
International Association for the Evaluation of Educational Achievement (IEA). 
Over the past 50 years, IEA has conducted comparative research studies focusing 
on educational policies, practices, and outcomes in more than 80 countries 
around the world. 

The purpose of ICCS is to investigate the ways in which young people in lower 
secondary schools are prepared to undertake their roles as citizens in a range of 
countries. The study will report on student achievement on a test of conceptual 
understandings and competencies in civic and citizenship education and also 
will collect and analyze data about student dispositions and attitudes relating 
to civic and citizenship education. Teacher and school questionnaires will gather 
information about teaching and class-management practices, school governance 
and climate, and other matters. A national context survey will collect information 
about civic and citizenship education and its contexts (aspects related to political, 
cultural, and educational contexts) in each participating country. 

The study framework also offers "regional modules" that will allow groups of 
participating countries from the same region to address region-specific issues in 
civic and citizenship education. The three regional modules established for ICCS 
relate to Europe, Latin America, and Asia. 




